2026-02-17T04:39:00.662 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T04:39:00.677 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T04:39:00.753 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-17_03:53:03-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial/50889 branch: wip-bharath18-testing-2026-02-13-0856 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '50889' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-17_03:53:03-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath18-testing-2026-02-13-0856 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 325850157d309f65985eea9ec9bb9f1d56726fee ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 325850157d309f65985eea9ec9bb9f1d56726fee workunit: branch: wip-bharath18-testing-2026-02-13-0856 sha1: 325850157d309f65985eea9ec9bb9f1d56726fee owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 7109 sha1: 325850157d309f65985eea9ec9bb9f1d56726fee sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath18-testing-2026-02-13-0856 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 325850157d309f65985eea9ec9bb9f1d56726fee targets: trial105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDoHOeK4CYnGRWmnJYagPgwLlkhvu+6KYM02PLxuzhh4krJbGxfgj4AWyYtRFo67YInreZYgLfmib4u+6kA27s= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_03:53:03 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T04:39:00.753 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa; will attempt to use it 2026-02-17T04:39:00.754 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks 2026-02-17T04:39:00.754 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T04:39:00.761 INFO:teuthology.task.internal:Checking packages... 2026-02-17T04:39:00.761 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '325850157d309f65985eea9ec9bb9f1d56726fee' 2026-02-17T04:39:00.761 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T04:39:00.761 INFO:teuthology.packaging:ref: None 2026-02-17T04:39:00.761 INFO:teuthology.packaging:tag: None 2026-02-17T04:39:00.761 INFO:teuthology.packaging:branch: wip-bharath18-testing-2026-02-13-0856 2026-02-17T04:39:00.761 INFO:teuthology.packaging:sha1: 325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-17T04:39:00.761 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath18-testing-2026-02-13-0856 2026-02-17T04:39:00.882 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5271-g32585015-1jammy 2026-02-17T04:39:00.884 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T04:39:00.903 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T04:39:00.904 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T04:39:00.917 INFO:teuthology.task.internal:Saving configuration 2026-02-17T04:39:00.922 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T04:39:00.934 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T04:39:00.995 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial105.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_03:53:03-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial/50889', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 04:35:32.704659', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFFWFA9cJO05ltaDzgL13Xii/6rsK9BQiWJ9KmKmEj4D'} 2026-02-17T04:39:00.995 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T04:39:01.008 INFO:teuthology.task.internal:roles: ubuntu@trial105.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T04:39:01.008 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T04:39:01.089 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x747ead5ebba0>, signals=[15]) 2026-02-17T04:39:01.089 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T04:39:01.100 INFO:teuthology.task.internal:Opening connections... 2026-02-17T04:39:01.100 DEBUG:teuthology.task.internal:connecting to ubuntu@trial105.front.sepia.ceph.com 2026-02-17T04:39:01.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T04:39:01.162 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T04:39:01.174 DEBUG:teuthology.orchestra.run.trial105:> uname -m 2026-02-17T04:39:01.241 INFO:teuthology.orchestra.run.trial105.stdout:x86_64 2026-02-17T04:39:01.242 DEBUG:teuthology.orchestra.run.trial105:> cat /etc/os-release 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:NAME="Ubuntu" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:VERSION_ID="22.04" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:VERSION_CODENAME=jammy 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:ID=ubuntu 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:ID_LIKE=debian 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-17T04:39:01.291 INFO:teuthology.orchestra.run.trial105.stdout:UBUNTU_CODENAME=jammy 2026-02-17T04:39:01.292 INFO:teuthology.lock.ops:Updating trial105.front.sepia.ceph.com on lock server 2026-02-17T04:39:01.370 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T04:39:01.382 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T04:39:01.394 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T04:39:01.394 DEBUG:teuthology.orchestra.run.trial105:> test '!' -e /home/ubuntu/cephtest 2026-02-17T04:39:01.398 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T04:39:01.409 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T04:39:01.409 DEBUG:teuthology.orchestra.run.trial105:> test -z $(ls -A /var/lib/ceph) 2026-02-17T04:39:01.443 INFO:teuthology.orchestra.run.trial105.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-17T04:39:01.444 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T04:39:01.541 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T04:39:01.559 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T04:39:01.559 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T04:39:01.559 DEBUG:teuthology.orchestra.run.trial105:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T04:39:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T04:39:01.562 DEBUG:teuthology.orchestra.run.trial105:> uname -r 2026-02-17T04:39:01.607 INFO:teuthology.orchestra.run.trial105.stdout:5.15.0-164-generic 2026-02-17T04:39:01.607 INFO:teuthology.task.kernel:Running kernel on trial105: 5.15.0-164-generic 2026-02-17T04:39:01.607 DEBUG:teuthology.orchestra.run.trial105:> sudo apt-get clean 2026-02-17T04:39:01.665 DEBUG:teuthology.orchestra.run.trial105:> sudo apt-get update 2026-02-17T04:39:01.853 INFO:teuthology.orchestra.run.trial105.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T04:39:01.860 INFO:teuthology.orchestra.run.trial105.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-17T04:39:01.896 INFO:teuthology.orchestra.run.trial105.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-17T04:39:01.911 INFO:teuthology.orchestra.run.trial105.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-17T04:39:01.995 INFO:teuthology.orchestra.run.trial105.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3237 kB] 2026-02-17T04:39:02.057 INFO:teuthology.orchestra.run.trial105.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [493 kB] 2026-02-17T04:39:02.060 INFO:teuthology.orchestra.run.trial105.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-17T04:39:02.061 INFO:teuthology.orchestra.run.trial105.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1255 kB] 2026-02-17T04:39:02.068 INFO:teuthology.orchestra.run.trial105.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [313 kB] 2026-02-17T04:39:02.070 INFO:teuthology.orchestra.run.trial105.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-17T04:39:02.070 INFO:teuthology.orchestra.run.trial105.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5287 kB] 2026-02-17T04:39:02.099 INFO:teuthology.orchestra.run.trial105.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [997 kB] 2026-02-17T04:39:02.102 INFO:teuthology.orchestra.run.trial105.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2966 kB] 2026-02-17T04:39:02.111 INFO:teuthology.orchestra.run.trial105.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [426 kB] 2026-02-17T04:39:02.112 INFO:teuthology.orchestra.run.trial105.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-17T04:39:02.112 INFO:teuthology.orchestra.run.trial105.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1017 kB] 2026-02-17T04:39:02.115 INFO:teuthology.orchestra.run.trial105.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [224 kB] 2026-02-17T04:39:02.116 INFO:teuthology.orchestra.run.trial105.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.6 kB] 2026-02-17T04:39:02.116 INFO:teuthology.orchestra.run.trial105.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5096 kB] 2026-02-17T04:39:02.132 INFO:teuthology.orchestra.run.trial105.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [964 kB] 2026-02-17T04:39:05.859 INFO:teuthology.orchestra.run.trial105.stdout:Fetched 22.7 MB in 2s (13.4 MB/s) 2026-02-17T04:39:06.396 INFO:teuthology.orchestra.run.trial105.stdout:Reading package lists... 2026-02-17T04:39:06.410 DEBUG:teuthology.orchestra.run.trial105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T04:39:06.443 INFO:teuthology.orchestra.run.trial105.stdout:Reading package lists... 2026-02-17T04:39:06.550 INFO:teuthology.orchestra.run.trial105.stdout:Building dependency tree... 2026-02-17T04:39:06.551 INFO:teuthology.orchestra.run.trial105.stdout:Reading state information... 2026-02-17T04:39:06.636 INFO:teuthology.orchestra.run.trial105.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T04:39:06.636 INFO:teuthology.orchestra.run.trial105.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T04:39:06.636 INFO:teuthology.orchestra.run.trial105.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T04:39:06.636 INFO:teuthology.orchestra.run.trial105.stdout: sg3-utils-udev 2026-02-17T04:39:06.636 INFO:teuthology.orchestra.run.trial105.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T04:39:06.637 INFO:teuthology.orchestra.run.trial105.stdout:The following additional packages will be installed: 2026-02-17T04:39:06.637 INFO:teuthology.orchestra.run.trial105.stdout: linux-generic linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T04:39:06.637 INFO:teuthology.orchestra.run.trial105.stdout: linux-headers-generic linux-image-5.15.0-170-generic 2026-02-17T04:39:06.637 INFO:teuthology.orchestra.run.trial105.stdout: linux-modules-5.15.0-170-generic linux-modules-extra-5.15.0-170-generic 2026-02-17T04:39:06.637 INFO:teuthology.orchestra.run.trial105.stdout:Suggested packages: 2026-02-17T04:39:06.637 INFO:teuthology.orchestra.run.trial105.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-17T04:39:06.650 INFO:teuthology.orchestra.run.trial105.stdout:The following NEW packages will be installed: 2026-02-17T04:39:06.650 INFO:teuthology.orchestra.run.trial105.stdout: linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T04:39:06.650 INFO:teuthology.orchestra.run.trial105.stdout: linux-image-5.15.0-170-generic linux-modules-5.15.0-170-generic 2026-02-17T04:39:06.651 INFO:teuthology.orchestra.run.trial105.stdout: linux-modules-extra-5.15.0-170-generic 2026-02-17T04:39:06.651 INFO:teuthology.orchestra.run.trial105.stdout:The following packages will be upgraded: 2026-02-17T04:39:06.651 INFO:teuthology.orchestra.run.trial105.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-17T04:39:06.747 INFO:teuthology.orchestra.run.trial105.stdout:3 upgraded, 5 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T04:39:06.748 INFO:teuthology.orchestra.run.trial105.stdout:Need to get 113 MB of archives. 2026-02-17T04:39:06.748 INFO:teuthology.orchestra.run.trial105.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-17T04:39:06.748 INFO:teuthology.orchestra.run.trial105.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-170-generic amd64 5.15.0-170.180 [22.7 MB] 2026-02-17T04:39:06.914 INFO:teuthology.orchestra.run.trial105.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-170-generic amd64 5.15.0-170.180 [11.6 MB] 2026-02-17T04:39:06.958 INFO:teuthology.orchestra.run.trial105.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-170-generic amd64 5.15.0-170.180 [64.0 MB] 2026-02-17T04:39:07.196 INFO:teuthology.orchestra.run.trial105.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.170.159 [1654 B] 2026-02-17T04:39:07.197 INFO:teuthology.orchestra.run.trial105.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.170.159 [2474 B] 2026-02-17T04:39:07.197 INFO:teuthology.orchestra.run.trial105.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170 all 5.15.0-170.180 [12.4 MB] 2026-02-17T04:39:07.244 INFO:teuthology.orchestra.run.trial105.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170-generic amd64 5.15.0-170.180 [2828 kB] 2026-02-17T04:39:07.255 INFO:teuthology.orchestra.run.trial105.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.170.159 [2316 B] 2026-02-17T04:39:07.451 INFO:teuthology.orchestra.run.trial105.stdout:Fetched 113 MB in 1s (188 MB/s) 2026-02-17T04:39:07.468 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package linux-modules-5.15.0-170-generic. 2026-02-17T04:39:07.635 INFO:teuthology.orchestra.run.trial105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 131954 files and directories currently installed.) 2026-02-17T04:39:07.636 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../0-linux-modules-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T04:39:07.637 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:07.990 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package linux-image-5.15.0-170-generic. 2026-02-17T04:39:07.997 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../1-linux-image-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T04:39:08.019 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:08.058 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-170-generic. 2026-02-17T04:39:08.064 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T04:39:08.065 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:09.168 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../3-linux-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T04:39:09.169 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T04:39:09.190 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T04:39:09.190 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-image-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T04:39:09.206 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package linux-headers-5.15.0-170. 2026-02-17T04:39:09.213 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../5-linux-headers-5.15.0-170_5.15.0-170.180_all.deb ... 2026-02-17T04:39:09.213 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T04:39:10.563 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package linux-headers-5.15.0-170-generic. 2026-02-17T04:39:10.576 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../6-linux-headers-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T04:39:10.576 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:11.155 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T04:39:11.156 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking linux-headers-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T04:39:11.181 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T04:39:11.182 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:11.183 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-headers-generic (5.15.0.170.159) ... 2026-02-17T04:39:11.184 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:11.975 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:12.825 INFO:teuthology.orchestra.run.trial105.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-17T04:39:12.825 INFO:teuthology.orchestra.run.trial105.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-170-generic 2026-02-17T04:39:12.832 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:13.640 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-image-generic (5.15.0.170.159) ... 2026-02-17T04:39:13.641 INFO:teuthology.orchestra.run.trial105.stdout:Setting up linux-generic (5.15.0.170.159) ... 2026-02-17T04:39:13.642 INFO:teuthology.orchestra.run.trial105.stdout:Processing triggers for linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T04:39:13.647 INFO:teuthology.orchestra.run.trial105.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-17T04:39:13.647 INFO:teuthology.orchestra.run.trial105.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-170-generic 2026-02-17T04:39:20.705 INFO:teuthology.orchestra.run.trial105.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-17T04:39:27.269 INFO:teuthology.orchestra.run.trial105.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-17T04:39:27.270 INFO:teuthology.orchestra.run.trial105.stdout:Sourcing file `/etc/default/grub' 2026-02-17T04:39:27.288 INFO:teuthology.orchestra.run.trial105.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T04:39:27.289 INFO:teuthology.orchestra.run.trial105.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T04:39:27.291 INFO:teuthology.orchestra.run.trial105.stdout:Generating grub configuration file ... 2026-02-17T04:39:27.371 INFO:teuthology.orchestra.run.trial105.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T04:39:27.378 INFO:teuthology.orchestra.run.trial105.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T04:39:27.550 INFO:teuthology.orchestra.run.trial105.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T04:39:27.556 INFO:teuthology.orchestra.run.trial105.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T04:39:27.629 INFO:teuthology.orchestra.run.trial105.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T04:39:27.629 INFO:teuthology.orchestra.run.trial105.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T04:39:27.629 INFO:teuthology.orchestra.run.trial105.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T04:39:27.631 INFO:teuthology.orchestra.run.trial105.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T04:39:27.640 INFO:teuthology.orchestra.run.trial105.stdout:done 2026-02-17T04:39:28.051 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.051 INFO:teuthology.orchestra.run.trial105.stdout:Pending kernel upgrade! 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout:Running kernel version: 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout: 5.15.0-164-generic 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout:Diagnostics: 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout: The currently running kernel version is not the expected kernel version 5.15.0-170-generic. 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.052 INFO:teuthology.orchestra.run.trial105.stdout:The processor microcode seems to be up-to-date. 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout:No services need to be restarted. 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout:No containers need to be restarted. 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout:No user sessions are running outdated binaries. 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:39:28.053 INFO:teuthology.orchestra.run.trial105.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T04:39:28.718 DEBUG:teuthology.orchestra.run.trial105:> dpkg -s linux-image-generic 2026-02-17T04:39:28.769 INFO:teuthology.orchestra.run.trial105.stdout:Package: linux-image-generic 2026-02-17T04:39:28.769 INFO:teuthology.orchestra.run.trial105.stdout:Status: install ok installed 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Priority: optional 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Section: kernel 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Installed-Size: 21 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Architecture: amd64 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Source: linux-meta 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Version: 5.15.0.170.159 2026-02-17T04:39:28.770 INFO:teuthology.orchestra.run.trial105.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T04:39:28.771 INFO:teuthology.orchestra.run.trial105.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T04:39:28.771 INFO:teuthology.orchestra.run.trial105.stdout:Recommends: thermald 2026-02-17T04:39:28.771 INFO:teuthology.orchestra.run.trial105.stdout:Description: Generic Linux kernel image 2026-02-17T04:39:28.771 INFO:teuthology.orchestra.run.trial105.stdout: This package will always depend on the latest generic kernel image 2026-02-17T04:39:28.771 INFO:teuthology.orchestra.run.trial105.stdout: available. 2026-02-17T04:39:28.771 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-164-generic Expected: 5.15.0-170-generic 2026-02-17T04:39:28.771 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T04:39:28.772 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T04:39:28.772 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T04:39:28.772 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial105.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T04:39:28.772 DEBUG:teuthology.orchestra.run.trial105:> sudo apt-get clean 2026-02-17T04:39:28.851 DEBUG:teuthology.orchestra.run.trial105:> sudo apt-get update 2026-02-17T04:39:29.029 INFO:teuthology.orchestra.run.trial105.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T04:39:29.034 INFO:teuthology.orchestra.run.trial105.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T04:39:29.043 INFO:teuthology.orchestra.run.trial105.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T04:39:29.052 INFO:teuthology.orchestra.run.trial105.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T04:39:29.871 INFO:teuthology.orchestra.run.trial105.stdout:Reading package lists... 2026-02-17T04:39:29.885 DEBUG:teuthology.orchestra.run.trial105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T04:39:29.918 INFO:teuthology.orchestra.run.trial105.stdout:Reading package lists... 2026-02-17T04:39:30.029 INFO:teuthology.orchestra.run.trial105.stdout:Building dependency tree... 2026-02-17T04:39:30.029 INFO:teuthology.orchestra.run.trial105.stdout:Reading state information... 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout:linux-image-generic set to manually installed. 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout: sg3-utils-udev 2026-02-17T04:39:30.119 INFO:teuthology.orchestra.run.trial105.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T04:39:30.140 INFO:teuthology.orchestra.run.trial105.stdout:0 upgraded, 0 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T04:39:30.141 DEBUG:teuthology.orchestra.run.trial105:> dpkg -s linux-image-generic 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Package: linux-image-generic 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Status: install ok installed 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Priority: optional 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Section: kernel 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Installed-Size: 21 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Architecture: amd64 2026-02-17T04:39:30.191 INFO:teuthology.orchestra.run.trial105.stdout:Source: linux-meta 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout:Version: 5.15.0.170.159 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout:Recommends: thermald 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout:Description: Generic Linux kernel image 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout: This package will always depend on the latest generic kernel image 2026-02-17T04:39:30.192 INFO:teuthology.orchestra.run.trial105.stdout: available. 2026-02-17T04:39:30.192 DEBUG:teuthology.orchestra.run.trial105:> mktemp 2026-02-17T04:39:30.239 INFO:teuthology.orchestra.run.trial105.stdout:/tmp/tmp.3knXQtQVZy 2026-02-17T04:39:30.239 DEBUG:teuthology.orchestra.run.trial105:> sudo cp /boot/grub/grub.cfg /tmp/tmp.3knXQtQVZy 2026-02-17T04:39:30.287 DEBUG:teuthology.orchestra.run.trial105:> sudo chmod 0666 /tmp/tmp.3knXQtQVZy 2026-02-17T04:39:30.386 DEBUG:teuthology.orchestra.remote:trial105:/tmp/tmp.3knXQtQVZy is 7KB 2026-02-17T04:39:30.394 DEBUG:teuthology.orchestra.run.trial105:> rm -fr /tmp/tmp.3knXQtQVZy 2026-02-17T04:39:30.399 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-17T04:39:30.448 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:39:30.448 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-17T04:39:30.448 DEBUG:teuthology.orchestra.run.trial105:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-17T04:39:30.504 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-170-generic 2026-02-17T04:39:30.504 DEBUG:teuthology.orchestra.run.trial105:> sudo update-grub 2026-02-17T04:39:30.732 INFO:teuthology.orchestra.run.trial105.stderr:Sourcing file `/etc/default/grub' 2026-02-17T04:39:30.751 INFO:teuthology.orchestra.run.trial105.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T04:39:30.752 INFO:teuthology.orchestra.run.trial105.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T04:39:30.754 INFO:teuthology.orchestra.run.trial105.stderr:Generating grub configuration file ... 2026-02-17T04:39:30.834 INFO:teuthology.orchestra.run.trial105.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T04:39:30.840 INFO:teuthology.orchestra.run.trial105.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T04:39:31.012 INFO:teuthology.orchestra.run.trial105.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T04:39:31.018 INFO:teuthology.orchestra.run.trial105.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T04:39:31.086 INFO:teuthology.orchestra.run.trial105.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T04:39:31.086 INFO:teuthology.orchestra.run.trial105.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T04:39:31.086 INFO:teuthology.orchestra.run.trial105.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T04:39:31.088 INFO:teuthology.orchestra.run.trial105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T04:39:31.093 INFO:teuthology.orchestra.run.trial105.stderr:done 2026-02-17T04:39:31.095 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial105 2026-02-17T04:39:31.095 DEBUG:teuthology.orchestra.run.trial105:> sudo bash -c ' 2026-02-17T04:39:31.095 DEBUG:teuthology.orchestra.run.trial105:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T04:39:31.095 DEBUG:teuthology.orchestra.run.trial105:> ' 2026-02-17T04:40:01.097 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T04:40:01.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial105.front.sepia.ceph.com' 2026-02-17T04:40:01.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T04:41:01.159 DEBUG:teuthology.orchestra.remote:timed out 2026-02-17T04:41:10.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial105.front.sepia.ceph.com' 2026-02-17T04:41:10.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T04:41:10.298 DEBUG:teuthology.orchestra.run.trial105:> true 2026-02-17T04:41:10.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial105.front.sepia.ceph.com' 2026-02-17T04:41:10.689 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-170-generic"... 2026-02-17T04:41:10.689 DEBUG:teuthology.orchestra.run.trial105:> uname -r 2026-02-17T04:41:10.734 INFO:teuthology.orchestra.run.trial105.stdout:5.15.0-170-generic 2026-02-17T04:41:10.734 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-170-generic vs 5.15.0-170-generic 2026-02-17T04:41:10.734 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T04:41:10.734 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-17T04:41:11.735 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T04:41:11.735 DEBUG:teuthology.orchestra.run.trial105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T04:41:11.748 INFO:teuthology.orchestra.run.trial105.stdout:ttyS1 2026-02-17T04:41:11.776 DEBUG:teuthology.parallel:result is None 2026-02-17T04:41:11.776 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T04:41:11.787 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T04:41:11.787 DEBUG:teuthology.orchestra.run.trial105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T04:41:11.822 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T04:41:11.834 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T04:41:11.847 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T04:41:11.848 DEBUG:teuthology.orchestra.run.trial105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T04:41:11.876 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T04:41:11.891 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T04:41:11.892 DEBUG:teuthology.orchestra.run.trial105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T04:41:11.919 INFO:teuthology.orchestra.run.trial105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:41:11.925 INFO:teuthology.orchestra.run.trial105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:41:11.927 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T04:41:11.938 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T04:41:11.939 DEBUG:teuthology.orchestra.run.trial105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T04:41:11.979 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T04:41:11.996 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T04:41:11.997 DEBUG:teuthology.orchestra.run.trial105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T04:41:12.026 DEBUG:teuthology.orchestra.run.trial105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T04:41:12.070 DEBUG:teuthology.orchestra.run.trial105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T04:41:12.114 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:41:12.115 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T04:41:12.164 DEBUG:teuthology.orchestra.run.trial105:> sudo service rsyslog restart 2026-02-17T04:41:12.228 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T04:41:12.241 INFO:teuthology.task.internal:Starting timer... 2026-02-17T04:41:12.242 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T04:41:12.256 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T04:41:12.270 DEBUG:teuthology.task.selinux:Excluding trial105: OS 'ubuntu' does not support SELinux 2026-02-17T04:41:12.271 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T04:41:12.271 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-17T04:41:12.271 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T04:41:12.271 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T04:41:12.284 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T04:41:12.294 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T04:41:12.295 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T04:43:31.643 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial105.front.sepia.ceph.com')] 2026-02-17T04:43:31.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial105.front.sepia.ceph.com' 2026-02-17T04:43:31.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T04:43:31.712 DEBUG:teuthology.orchestra.run.trial105:> true 2026-02-17T04:43:31.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial105.front.sepia.ceph.com' 2026-02-17T04:43:31.958 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T04:43:31.970 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T04:43:31.970 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T04:43:31.971 DEBUG:teuthology.orchestra.run.trial105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Command line: ntpd -gq 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: ---------------------------------------------------- 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: ntp-4 is maintained by Network Time Foundation, 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: corporation. Support and training for ntp-4 are 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: available at https://www.nwtime.org/support 2026-02-17T04:43:32.019 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: ---------------------------------------------------- 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: proto: precision = 0.020 usec (-25) 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: basedate set to 2022-02-04 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: gps base set to 2022-02-06 (week 2196) 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: restrict ::: KOD does nothing without LIMITED. 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listen and drop on 0 v6wildcard [::]:123 2026-02-17T04:43:32.020 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-17T04:43:32.021 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listen normally on 2 lo 127.0.0.1:123 2026-02-17T04:43:32.021 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listen normally on 3 enp1s0f0 10.20.193.105:123 2026-02-17T04:43:32.021 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listen normally on 4 lo [::1]:123 2026-02-17T04:43:32.021 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:62be%2]:123 2026-02-17T04:43:32.021 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:32 ntpd[8771]: Listening on routing socket on fd #22 for interface updates 2026-02-17T04:43:33.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:33 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:33.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:33 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:33.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:33 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:33.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:33 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:33.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:33 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:33.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:33 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:34.019 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:34 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:34.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:34 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:34.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:34 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:34.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:34 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:34.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:34 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:34.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:34 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:35.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:35 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:35.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:35 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:35.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:35 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:35.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:35 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:35.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:35 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:35.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:35 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:36.019 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:36.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:36.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:36.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:36.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:36.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:36.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:36.033 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:36.084 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:36.084 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:36.084 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:36.084 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:36 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:37.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:37.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:37.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:37.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:37.087 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:37.087 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:37.087 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:37.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:37.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:37.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:37.089 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:37.089 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:37 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:38.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:38.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:38.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:38.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:38.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:38.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:38.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:38.032 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:38.083 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:38.083 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:38.084 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:38.084 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:38 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:39.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:39.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:39.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:39.020 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:39.086 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:39.086 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:39.086 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:39.086 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:39.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T04:43:39.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T04:43:39.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T04:43:39.088 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T04:43:39.089 INFO:teuthology.orchestra.run.trial105.stdout:17 Feb 04:43:39 ntpd[8771]: ntpd: time set -0.193889 s 2026-02-17T04:43:39.089 INFO:teuthology.orchestra.run.trial105.stdout:ntpd: time set -0.193889s 2026-02-17T04:43:39.089 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-17T04:43:39.090 INFO:teuthology.orchestra.run.trial105.stderr:17 Feb 04:43:39 ntpd[8771]: can't open /var/log/ntpstats/loopstats.20260217: Permission denied 2026-02-17T04:43:39.154 INFO:teuthology.orchestra.run.trial105.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T04:43:39.154 INFO:teuthology.orchestra.run.trial105.stdout:============================================================================== 2026-02-17T04:43:39.154 INFO:teuthology.orchestra.run.trial105.stdout: lax1.us.ntp.li .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T04:43:39.154 INFO:teuthology.orchestra.run.trial105.stdout: baron.cgee.net .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T04:43:39.154 INFO:teuthology.orchestra.run.trial105.stdout: h96-60-247-29.m .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T04:43:39.154 INFO:teuthology.orchestra.run.trial105.stdout: ntp.maxhost.io .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T04:43:39.155 INFO:teuthology.run_tasks:Running task install... 2026-02-17T04:43:39.166 DEBUG:teuthology.task.install:project ceph 2026-02-17T04:43:39.166 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '325850157d309f65985eea9ec9bb9f1d56726fee'}} 2026-02-17T04:43:39.166 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '325850157d309f65985eea9ec9bb9f1d56726fee'} 2026-02-17T04:43:39.166 INFO:teuthology.task.install:Using flavor: default 2026-02-17T04:43:39.169 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T04:43:39.169 INFO:teuthology.task.install:extra packages: [] 2026-02-17T04:43:39.170 DEBUG:teuthology.orchestra.run.trial105:> sudo apt-key list | grep Ceph 2026-02-17T04:43:39.249 INFO:teuthology.orchestra.run.trial105.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-17T04:43:39.279 INFO:teuthology.orchestra.run.trial105.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-17T04:43:39.279 INFO:teuthology.orchestra.run.trial105.stdout:uid [ unknown] Ceph.com (release key) 2026-02-17T04:43:39.280 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-17T04:43:39.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-17T04:43:39.409 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default/ 2026-02-17T04:43:39.409 INFO:teuthology.task.install.deb:Package version is 20.3.0-5271-g32585015-1jammy 2026-02-17T04:43:39.541 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:43:39.541 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-17T04:43:39.552 DEBUG:teuthology.orchestra.run.trial105:> sudo apt-get update 2026-02-17T04:43:39.706 INFO:teuthology.orchestra.run.trial105.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy InRelease 2026-02-17T04:43:39.717 INFO:teuthology.orchestra.run.trial105.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-17T04:43:39.729 INFO:teuthology.orchestra.run.trial105.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-17T04:43:39.740 INFO:teuthology.orchestra.run.trial105.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-17T04:43:39.986 INFO:teuthology.orchestra.run.trial105.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T04:43:40.053 INFO:teuthology.orchestra.run.trial105.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T04:43:40.125 INFO:teuthology.orchestra.run.trial105.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T04:43:40.197 INFO:teuthology.orchestra.run.trial105.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T04:43:40.343 INFO:teuthology.orchestra.run.trial105.stdout:Fetched 26.9 kB in 1s (42.4 kB/s) 2026-02-17T04:43:40.870 INFO:teuthology.orchestra.run.trial105.stdout:Reading package lists... 2026-02-17T04:43:40.883 DEBUG:teuthology.orchestra.run.trial105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5271-g32585015-1jammy cephadm=20.3.0-5271-g32585015-1jammy ceph-mds=20.3.0-5271-g32585015-1jammy ceph-mgr=20.3.0-5271-g32585015-1jammy ceph-osd=20.3.0-5271-g32585015-1jammy ceph-osd-classic=20.3.0-5271-g32585015-1jammy ceph-common=20.3.0-5271-g32585015-1jammy ceph-fuse=20.3.0-5271-g32585015-1jammy ceph-test=20.3.0-5271-g32585015-1jammy ceph-volume=20.3.0-5271-g32585015-1jammy radosgw=20.3.0-5271-g32585015-1jammy python3-rados=20.3.0-5271-g32585015-1jammy python3-rgw=20.3.0-5271-g32585015-1jammy python3-cephfs=20.3.0-5271-g32585015-1jammy python3-rbd=20.3.0-5271-g32585015-1jammy libcephfs2=20.3.0-5271-g32585015-1jammy libcephfs-dev=20.3.0-5271-g32585015-1jammy librados2=20.3.0-5271-g32585015-1jammy librbd1=20.3.0-5271-g32585015-1jammy rbd-fuse=20.3.0-5271-g32585015-1jammy 2026-02-17T04:43:40.917 INFO:teuthology.orchestra.run.trial105.stdout:Reading package lists... 2026-02-17T04:43:41.026 INFO:teuthology.orchestra.run.trial105.stdout:Building dependency tree... 2026-02-17T04:43:41.026 INFO:teuthology.orchestra.run.trial105.stdout:Reading state information... 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: sg3-utils-udev 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout:The following additional packages will be installed: 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-17T04:43:41.115 INFO:teuthology.orchestra.run.trial105.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout:Suggested packages: 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout:Recommended packages: 2026-02-17T04:43:41.116 INFO:teuthology.orchestra.run.trial105.stdout: btrfs-tools 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout:The following NEW packages will be installed: 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T04:43:41.150 INFO:teuthology.orchestra.run.trial105.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-17T04:43:41.151 INFO:teuthology.orchestra.run.trial105.stdout: socat xmlstarlet 2026-02-17T04:43:41.228 INFO:teuthology.orchestra.run.trial105.stdout:0 upgraded, 87 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T04:43:41.228 INFO:teuthology.orchestra.run.trial105.stdout:Need to get 702 MB of archives. 2026-02-17T04:43:41.228 INFO:teuthology.orchestra.run.trial105.stdout:After this operation, 2670 MB of additional disk space will be used. 2026-02-17T04:43:41.228 INFO:teuthology.orchestra.run.trial105.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5271-g32585015-1jammy [3799 kB] 2026-02-17T04:43:41.333 INFO:teuthology.orchestra.run.trial105.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5271-g32585015-1jammy [2910 kB] 2026-02-17T04:43:41.372 INFO:teuthology.orchestra.run.trial105.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5271-g32585015-1jammy [882 kB] 2026-02-17T04:43:41.386 INFO:teuthology.orchestra.run.trial105.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5271-g32585015-1jammy [359 kB] 2026-02-17T04:43:41.392 INFO:teuthology.orchestra.run.trial105.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5271-g32585015-1jammy [32.8 kB] 2026-02-17T04:43:41.392 INFO:teuthology.orchestra.run.trial105.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5271-g32585015-1jammy [225 kB] 2026-02-17T04:43:41.395 INFO:teuthology.orchestra.run.trial105.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5271-g32585015-1jammy [88.4 kB] 2026-02-17T04:43:41.397 INFO:teuthology.orchestra.run.trial105.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5271-g32585015-1jammy [339 kB] 2026-02-17T04:43:41.403 INFO:teuthology.orchestra.run.trial105.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5271-g32585015-1jammy [9091 kB] 2026-02-17T04:43:41.506 INFO:teuthology.orchestra.run.trial105.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-17T04:43:41.553 INFO:teuthology.orchestra.run.trial105.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5271-g32585015-1jammy [112 kB] 2026-02-17T04:43:41.554 INFO:teuthology.orchestra.run.trial105.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5271-g32585015-1jammy [261 kB] 2026-02-17T04:43:41.558 INFO:teuthology.orchestra.run.trial105.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5271-g32585015-1jammy [31.5 MB] 2026-02-17T04:43:41.845 INFO:teuthology.orchestra.run.trial105.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-17T04:43:41.855 INFO:teuthology.orchestra.run.trial105.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-17T04:43:41.925 INFO:teuthology.orchestra.run.trial105.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-17T04:43:42.080 INFO:teuthology.orchestra.run.trial105.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5271-g32585015-1jammy [4983 kB] 2026-02-17T04:43:42.126 INFO:teuthology.orchestra.run.trial105.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-17T04:43:42.135 INFO:teuthology.orchestra.run.trial105.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-17T04:43:42.156 INFO:teuthology.orchestra.run.trial105.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5271-g32585015-1jammy [293 kB] 2026-02-17T04:43:42.160 INFO:teuthology.orchestra.run.trial105.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5271-g32585015-1jammy [127 kB] 2026-02-17T04:43:42.162 INFO:teuthology.orchestra.run.trial105.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-17T04:43:42.163 INFO:teuthology.orchestra.run.trial105.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5271-g32585015-1jammy [922 kB] 2026-02-17T04:43:42.171 INFO:teuthology.orchestra.run.trial105.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-17T04:43:42.173 INFO:teuthology.orchestra.run.trial105.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-17T04:43:42.174 INFO:teuthology.orchestra.run.trial105.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-17T04:43:42.174 INFO:teuthology.orchestra.run.trial105.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-17T04:43:42.177 INFO:teuthology.orchestra.run.trial105.stdout:Get:28 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5271-g32585015-1jammy [7537 kB] 2026-02-17T04:43:42.191 INFO:teuthology.orchestra.run.trial105.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-17T04:43:42.192 INFO:teuthology.orchestra.run.trial105.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-17T04:43:42.198 INFO:teuthology.orchestra.run.trial105.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-17T04:43:42.265 INFO:teuthology.orchestra.run.trial105.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-17T04:43:42.265 INFO:teuthology.orchestra.run.trial105.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-17T04:43:42.265 INFO:teuthology.orchestra.run.trial105.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-17T04:43:42.266 INFO:teuthology.orchestra.run.trial105.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-17T04:43:42.266 INFO:teuthology.orchestra.run.trial105.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-17T04:43:42.266 INFO:teuthology.orchestra.run.trial105.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-17T04:43:42.266 INFO:teuthology.orchestra.run.trial105.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-17T04:43:42.270 INFO:teuthology.orchestra.run.trial105.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-17T04:43:42.270 INFO:teuthology.orchestra.run.trial105.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-17T04:43:42.302 INFO:teuthology.orchestra.run.trial105.stdout:Get:41 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5271-g32585015-1jammy [468 MB] 2026-02-17T04:43:42.340 INFO:teuthology.orchestra.run.trial105.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-17T04:43:42.342 INFO:teuthology.orchestra.run.trial105.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-17T04:43:42.343 INFO:teuthology.orchestra.run.trial105.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-17T04:43:42.343 INFO:teuthology.orchestra.run.trial105.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-17T04:43:42.369 INFO:teuthology.orchestra.run.trial105.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-17T04:43:42.370 INFO:teuthology.orchestra.run.trial105.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-17T04:43:42.371 INFO:teuthology.orchestra.run.trial105.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-17T04:43:42.429 INFO:teuthology.orchestra.run.trial105.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-17T04:43:42.429 INFO:teuthology.orchestra.run.trial105.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-17T04:43:42.429 INFO:teuthology.orchestra.run.trial105.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-17T04:43:42.430 INFO:teuthology.orchestra.run.trial105.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-17T04:43:42.430 INFO:teuthology.orchestra.run.trial105.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-17T04:43:42.430 INFO:teuthology.orchestra.run.trial105.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-17T04:43:42.488 INFO:teuthology.orchestra.run.trial105.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-17T04:43:42.489 INFO:teuthology.orchestra.run.trial105.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-17T04:43:42.490 INFO:teuthology.orchestra.run.trial105.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-17T04:43:42.560 INFO:teuthology.orchestra.run.trial105.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-17T04:43:42.563 INFO:teuthology.orchestra.run.trial105.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-17T04:43:42.565 INFO:teuthology.orchestra.run.trial105.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-17T04:43:42.569 INFO:teuthology.orchestra.run.trial105.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-17T04:43:42.571 INFO:teuthology.orchestra.run.trial105.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-17T04:43:42.572 INFO:teuthology.orchestra.run.trial105.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-17T04:43:42.572 INFO:teuthology.orchestra.run.trial105.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-17T04:43:42.632 INFO:teuthology.orchestra.run.trial105.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-17T04:43:42.633 INFO:teuthology.orchestra.run.trial105.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-17T04:43:42.639 INFO:teuthology.orchestra.run.trial105.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-17T04:43:42.705 INFO:teuthology.orchestra.run.trial105.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-17T04:43:42.706 INFO:teuthology.orchestra.run.trial105.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-17T04:43:42.707 INFO:teuthology.orchestra.run.trial105.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-17T04:43:42.707 INFO:teuthology.orchestra.run.trial105.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-17T04:43:50.246 INFO:teuthology.orchestra.run.trial105.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5271-g32585015-1jammy [180 kB] 2026-02-17T04:43:50.247 INFO:teuthology.orchestra.run.trial105.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5271-g32585015-1jammy [14.1 kB] 2026-02-17T04:43:50.248 INFO:teuthology.orchestra.run.trial105.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5271-g32585015-1jammy [1030 kB] 2026-02-17T04:43:50.253 INFO:teuthology.orchestra.run.trial105.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5271-g32585015-1jammy [2437 kB] 2026-02-17T04:43:50.266 INFO:teuthology.orchestra.run.trial105.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5271-g32585015-1jammy [1069 kB] 2026-02-17T04:43:50.272 INFO:teuthology.orchestra.run.trial105.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5271-g32585015-1jammy [190 kB] 2026-02-17T04:43:50.273 INFO:teuthology.orchestra.run.trial105.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5271-g32585015-1jammy [54.7 MB] 2026-02-17T04:43:51.165 INFO:teuthology.orchestra.run.trial105.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5271-g32585015-1jammy [8626 kB] 2026-02-17T04:43:51.299 INFO:teuthology.orchestra.run.trial105.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5271-g32585015-1jammy [14.2 kB] 2026-02-17T04:43:51.299 INFO:teuthology.orchestra.run.trial105.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5271-g32585015-1jammy [68.6 MB] 2026-02-17T04:43:52.434 INFO:teuthology.orchestra.run.trial105.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5271-g32585015-1jammy [136 kB] 2026-02-17T04:43:52.435 INFO:teuthology.orchestra.run.trial105.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5271-g32585015-1jammy [45.1 kB] 2026-02-17T04:43:52.435 INFO:teuthology.orchestra.run.trial105.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5271-g32585015-1jammy [31.7 kB] 2026-02-17T04:43:52.435 INFO:teuthology.orchestra.run.trial105.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5271-g32585015-1jammy [44.2 kB] 2026-02-17T04:43:52.436 INFO:teuthology.orchestra.run.trial105.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5271-g32585015-1jammy [18.3 MB] 2026-02-17T04:43:52.729 INFO:teuthology.orchestra.run.trial105.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5271-g32585015-1jammy [93.7 kB] 2026-02-17T04:43:53.009 INFO:teuthology.orchestra.run.trial105.stdout:Fetched 702 MB in 12s (60.6 MB/s) 2026-02-17T04:43:53.026 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-17T04:43:53.181 INFO:teuthology.orchestra.run.trial105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167438 files and directories currently installed.) 2026-02-17T04:43:53.183 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-17T04:43:53.184 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T04:43:53.206 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-17T04:43:53.218 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-17T04:43:53.218 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T04:43:53.234 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-17T04:43:53.246 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-17T04:43:53.247 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T04:43:53.267 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-17T04:43:53.275 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T04:43:53.279 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T04:43:53.312 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-17T04:43:53.320 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T04:43:53.320 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T04:43:53.337 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-17T04:43:53.345 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T04:43:53.345 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T04:43:53.364 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-17T04:43:53.373 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-17T04:43:53.373 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T04:43:53.389 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package librados2. 2026-02-17T04:43:53.397 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../07-librados2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.398 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking librados2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.441 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libnbd0. 2026-02-17T04:43:53.450 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-17T04:43:53.450 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-17T04:43:53.465 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package librbd1. 2026-02-17T04:43:53.473 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../09-librbd1_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.473 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking librbd1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.510 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libcephfs2. 2026-02-17T04:43:53.519 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.519 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libcephfs2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.540 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-rados. 2026-02-17T04:43:53.547 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.547 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-rados (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.565 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-17T04:43:53.574 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:43:53.574 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-ceph-argparse (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.589 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-cephfs. 2026-02-17T04:43:53.596 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.597 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-cephfs (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.613 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-17T04:43:53.621 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:43:53.622 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.640 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-17T04:43:53.649 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-17T04:43:53.649 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T04:43:53.664 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-prettytable. 2026-02-17T04:43:53.673 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-17T04:43:53.673 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-17T04:43:53.687 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-rbd. 2026-02-17T04:43:53.695 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.695 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-rbd (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.713 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-17T04:43:53.721 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-17T04:43:53.722 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T04:43:53.739 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package librgw2. 2026-02-17T04:43:53.747 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../19-librgw2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.747 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking librgw2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.847 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-rgw. 2026-02-17T04:43:53.856 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.856 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-rgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.871 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-17T04:43:53.879 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-17T04:43:53.880 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T04:43:53.893 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libradosstriper1. 2026-02-17T04:43:53.902 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.902 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libradosstriper1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:53.919 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-common. 2026-02-17T04:43:53.927 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:53.927 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:54.192 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-base. 2026-02-17T04:43:54.200 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:54.206 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-base (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:54.265 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-17T04:43:54.273 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-17T04:43:54.273 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-17T04:43:54.288 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-cheroot. 2026-02-17T04:43:54.296 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-17T04:43:54.297 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T04:43:54.313 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-17T04:43:54.321 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-17T04:43:54.322 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-17T04:43:54.337 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-17T04:43:54.345 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-17T04:43:54.346 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-17T04:43:54.360 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-17T04:43:54.368 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-17T04:43:54.368 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-17T04:43:54.383 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-tempora. 2026-02-17T04:43:54.391 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-17T04:43:54.392 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-17T04:43:54.406 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-portend. 2026-02-17T04:43:54.414 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-17T04:43:54.414 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-17T04:43:54.428 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-17T04:43:54.437 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-17T04:43:54.437 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-17T04:43:54.452 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-17T04:43:54.460 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-17T04:43:54.460 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-17T04:43:54.484 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-natsort. 2026-02-17T04:43:54.492 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-17T04:43:54.493 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-17T04:43:54.509 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-17T04:43:54.518 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:43:54.518 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:54.548 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-17T04:43:54.556 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:54.556 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:54.571 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mgr. 2026-02-17T04:43:54.579 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:54.579 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mgr (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:54.600 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mon. 2026-02-17T04:43:54.608 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:54.608 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mon (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:54.675 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-17T04:43:54.683 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-17T04:43:54.684 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T04:43:54.698 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-17T04:43:54.706 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:54.707 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-osd-classic (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.629 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-osd. 2026-02-17T04:43:58.640 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:58.641 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-osd (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.660 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph. 2026-02-17T04:43:58.673 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../42-ceph_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:58.673 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.689 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-fuse. 2026-02-17T04:43:58.701 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:58.702 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.728 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mds. 2026-02-17T04:43:58.736 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:58.737 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mds (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.769 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package cephadm. 2026-02-17T04:43:58.777 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../45-cephadm_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:43:58.778 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.794 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-17T04:43:58.802 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T04:43:58.802 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T04:43:58.823 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-17T04:43:58.832 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:43:58.832 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:58.853 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-17T04:43:58.861 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-17T04:43:58.862 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-17T04:43:58.877 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-routes. 2026-02-17T04:43:58.885 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-17T04:43:58.885 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T04:43:58.905 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-17T04:43:58.913 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:43:58.913 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:59.587 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-17T04:43:59.595 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-17T04:43:59.596 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T04:43:59.634 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-joblib. 2026-02-17T04:43:59.643 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-17T04:43:59.643 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T04:43:59.668 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-17T04:43:59.677 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-17T04:43:59.677 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-17T04:43:59.692 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-sklearn. 2026-02-17T04:43:59.701 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-17T04:43:59.701 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T04:43:59.783 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-17T04:43:59.792 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:43:59.792 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:43:59.950 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-cachetools. 2026-02-17T04:43:59.958 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-17T04:43:59.959 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-17T04:43:59.973 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-rsa. 2026-02-17T04:43:59.982 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-17T04:43:59.982 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-17T04:43:59.999 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-google-auth. 2026-02-17T04:44:00.008 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-17T04:44:00.008 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-17T04:44:00.025 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-17T04:44:00.034 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-17T04:44:00.034 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T04:44:00.050 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-websocket. 2026-02-17T04:44:00.059 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-17T04:44:00.059 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-17T04:44:00.076 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-17T04:44:00.084 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-17T04:44:00.085 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T04:44:00.180 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-17T04:44:00.189 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:44:00.189 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:00.203 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-17T04:44:00.212 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-17T04:44:00.212 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T04:44:00.227 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-17T04:44:00.236 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T04:44:00.236 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T04:44:00.250 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package jq. 2026-02-17T04:44:00.259 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T04:44:00.259 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-17T04:44:00.274 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package socat. 2026-02-17T04:44:00.282 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-17T04:44:00.283 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-17T04:44:00.302 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package xmlstarlet. 2026-02-17T04:44:00.310 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-17T04:44:00.311 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-17T04:44:00.347 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-test. 2026-02-17T04:44:00.356 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:44:00.356 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-test (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:00.984 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package ceph-volume. 2026-02-17T04:44:00.993 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-17T04:44:00.994 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking ceph-volume (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:01.016 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-17T04:44:01.025 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:44:01.025 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libcephfs-daemon (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:01.039 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-17T04:44:01.048 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:44:01.048 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libcephfs-proxy2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:01.062 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-17T04:44:01.070 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:44:01.071 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking libcephfs-dev (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:01.086 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package nvme-cli. 2026-02-17T04:44:01.095 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-17T04:44:01.095 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T04:44:01.126 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-17T04:44:01.135 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T04:44:01.135 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T04:44:01.166 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-17T04:44:01.174 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-17T04:44:01.174 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-17T04:44:01.189 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-pluggy. 2026-02-17T04:44:01.198 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-17T04:44:01.198 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-17T04:44:01.213 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-psutil. 2026-02-17T04:44:01.222 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-17T04:44:01.222 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-17T04:44:01.240 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-py. 2026-02-17T04:44:01.249 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-17T04:44:01.249 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-17T04:44:01.269 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-pygments. 2026-02-17T04:44:01.277 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-17T04:44:01.277 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T04:44:01.319 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-toml. 2026-02-17T04:44:01.327 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-17T04:44:01.328 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-17T04:44:01.342 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-pytest. 2026-02-17T04:44:01.350 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-17T04:44:01.350 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T04:44:01.372 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-simplejson. 2026-02-17T04:44:01.381 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-17T04:44:01.382 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-17T04:44:01.407 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package python3-webob. 2026-02-17T04:44:01.415 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-17T04:44:01.415 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T04:44:01.431 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-17T04:44:01.440 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-17T04:44:01.440 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-17T04:44:01.508 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package radosgw. 2026-02-17T04:44:01.517 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../85-radosgw_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:44:01.517 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking radosgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:01.681 INFO:teuthology.orchestra.run.trial105.stdout:Selecting previously unselected package rbd-fuse. 2026-02-17T04:44:01.690 INFO:teuthology.orchestra.run.trial105.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-17T04:44:01.690 INFO:teuthology.orchestra.run.trial105.stdout:Unpacking rbd-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:01.718 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-17T04:44:01.785 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T04:44:01.786 INFO:teuthology.orchestra.run.trial105.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T04:44:01.856 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T04:44:02.126 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-17T04:44:02.517 INFO:teuthology.orchestra.run.trial105.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-17T04:44:02.527 INFO:teuthology.orchestra.run.trial105.stdout:Setting up cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:02.570 INFO:teuthology.orchestra.run.trial105.stdout:Adding system user cephadm....done 2026-02-17T04:44:02.576 INFO:teuthology.orchestra.run.trial105.stdout:usermod: no changes 2026-02-17T04:44:02.582 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-17T04:44:02.654 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T04:44:02.655 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-17T04:44:02.726 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-17T04:44:02.801 INFO:teuthology.orchestra.run.trial105.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T04:44:02.802 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-17T04:44:02.896 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T04:44:03.021 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-17T04:44:03.097 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-17T04:44:03.170 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-ceph-argparse (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:03.239 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T04:44:03.239 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-17T04:44:03.241 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T04:44:03.241 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T04:44:03.242 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-17T04:44:03.364 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-17T04:44:03.443 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libcephfs-proxy2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:03.444 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T04:44:03.523 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-17T04:44:03.612 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T04:44:03.866 INFO:teuthology.orchestra.run.trial105.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-17T04:44:03.867 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T04:44:03.964 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T04:44:04.092 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T04:44:04.175 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-17T04:44:04.246 INFO:teuthology.orchestra.run.trial105.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-17T04:44:04.247 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:04.337 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T04:44:04.845 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T04:44:04.849 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-17T04:44:04.926 INFO:teuthology.orchestra.run.trial105.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T04:44:04.927 INFO:teuthology.orchestra.run.trial105.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-17T04:44:04.928 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-17T04:44:05.005 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-17T04:44:05.077 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T04:44:05.078 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-17T04:44:05.156 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-17T04:44:05.228 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-17T04:44:05.308 INFO:teuthology.orchestra.run.trial105.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T04:44:05.309 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-17T04:44:05.393 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T04:44:05.394 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T04:44:05.470 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T04:44:05.554 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-17T04:44:05.627 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T04:44:05.628 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T04:44:05.765 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-17T04:44:05.838 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T04:44:05.839 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-17T04:44:05.922 INFO:teuthology.orchestra.run.trial105.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-17T04:44:05.923 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-17T04:44:06.053 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T04:44:06.054 INFO:teuthology.orchestra.run.trial105.stdout:Setting up librados2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.055 INFO:teuthology.orchestra.run.trial105.stdout:Setting up librgw2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.056 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.057 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T04:44:06.586 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libcephfs2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.587 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libradosstriper1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.588 INFO:teuthology.orchestra.run.trial105.stdout:Setting up librbd1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.589 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mgr-modules-core (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.590 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:06.662 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T04:44:06.662 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T04:44:07.066 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libcephfs-dev (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.067 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-rados (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.068 INFO:teuthology.orchestra.run.trial105.stdout:Setting up libcephfs-daemon (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.068 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-rbd (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.070 INFO:teuthology.orchestra.run.trial105.stdout:Setting up rbd-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.071 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-rgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.071 INFO:teuthology.orchestra.run.trial105.stdout:Setting up python3-cephfs (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.072 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.100 INFO:teuthology.orchestra.run.trial105.stdout:Adding group ceph....done 2026-02-17T04:44:07.137 INFO:teuthology.orchestra.run.trial105.stdout:Adding system user ceph....done 2026-02-17T04:44:07.148 INFO:teuthology.orchestra.run.trial105.stdout:Setting system user ceph properties....done 2026-02-17T04:44:07.154 INFO:teuthology.orchestra.run.trial105.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-17T04:44:07.241 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-17T04:44:07.509 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-17T04:44:07.904 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-test (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:07.905 INFO:teuthology.orchestra.run.trial105.stdout:Setting up radosgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:08.167 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T04:44:08.167 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T04:44:08.560 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-base (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:08.677 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-17T04:44:09.065 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mds (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:09.125 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T04:44:09.125 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T04:44:09.508 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mgr (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:09.588 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T04:44:09.588 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T04:44:09.959 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:09.960 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-osd-classic (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:09.977 INFO:teuthology.orchestra.run.trial105.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-17T04:44:09.987 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:09.997 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mon (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:10.068 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T04:44:10.068 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T04:44:10.462 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mgr-cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:10.473 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-mgr-dashboard (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:10.485 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-osd (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:10.571 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T04:44:10.572 INFO:teuthology.orchestra.run.trial105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T04:44:10.986 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:10.987 INFO:teuthology.orchestra.run.trial105.stdout:Setting up ceph-volume (20.3.0-5271-g32585015-1jammy) ... 2026-02-17T04:44:11.094 INFO:teuthology.orchestra.run.trial105.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-17T04:44:11.178 INFO:teuthology.orchestra.run.trial105.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.11) ... 2026-02-17T04:44:11.633 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:44:11.633 INFO:teuthology.orchestra.run.trial105.stdout:Running kernel seems to be up-to-date. 2026-02-17T04:44:11.633 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:44:11.633 INFO:teuthology.orchestra.run.trial105.stdout:The processor microcode seems to be up-to-date. 2026-02-17T04:44:11.633 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout:No services need to be restarted. 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout:No containers need to be restarted. 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout:No user sessions are running outdated binaries. 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout: 2026-02-17T04:44:11.634 INFO:teuthology.orchestra.run.trial105.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T04:44:12.282 INFO:teuthology.orchestra.run.trial105.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-17T04:44:12.287 DEBUG:teuthology.parallel:result is None 2026-02-17T04:44:12.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-17T04:44:12.427 DEBUG:teuthology.orchestra.run.trial105:> dpkg-query -W -f '${Version}' ceph 2026-02-17T04:44:12.438 INFO:teuthology.orchestra.run.trial105.stdout:20.3.0-5271-g32585015-1jammy 2026-02-17T04:44:12.438 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5271-g32585015-1jammy 2026-02-17T04:44:12.438 INFO:teuthology.task.install:The correct ceph version 20.3.0-5271-g32585015-1jammy is installed. 2026-02-17T04:44:12.441 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T04:44:12.441 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:44:12.441 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T04:44:12.489 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T04:44:12.489 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:44:12.490 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T04:44:12.539 DEBUG:teuthology.orchestra.run.trial105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T04:44:12.587 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T04:44:12.588 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:44:12.588 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T04:44:12.636 DEBUG:teuthology.orchestra.run.trial105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T04:44:12.684 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T04:44:12.684 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:44:12.685 DEBUG:teuthology.orchestra.run.trial105:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T04:44:12.734 DEBUG:teuthology.orchestra.run.trial105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T04:44:12.784 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T04:44:12.807 INFO:tasks.workunit:Pulling workunits from ref 325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-17T04:44:12.807 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T04:44:12.808 INFO:tasks.workunit:timeout=3h 2026-02-17T04:44:12.808 INFO:tasks.workunit:cleanup=True 2026-02-17T04:44:12.808 DEBUG:teuthology.orchestra.run.trial105:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T04:44:12.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T04:44:12.829 INFO:teuthology.orchestra.run.trial105.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T04:44:12.829 DEBUG:teuthology.orchestra.run.trial105:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T04:44:12.877 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T04:44:12.877 DEBUG:teuthology.orchestra.run.trial105:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T04:44:12.925 DEBUG:teuthology.orchestra.run.trial105:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-17T04:44:12.975 INFO:tasks.workunit.client.0.trial105.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T04:44:34.837 INFO:tasks.workunit.client.0.trial105.stderr:Note: switching to '325850157d309f65985eea9ec9bb9f1d56726fee'. 2026-02-17T04:44:34.837 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.837 INFO:tasks.workunit.client.0.trial105.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T04:44:34.837 INFO:tasks.workunit.client.0.trial105.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T04:44:34.837 INFO:tasks.workunit.client.0.trial105.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: git switch -c 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr:Or undo this operation with: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: git switch - 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.838 INFO:tasks.workunit.client.0.trial105.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T04:44:34.839 INFO:tasks.workunit.client.0.trial105.stderr: 2026-02-17T04:44:34.839 INFO:tasks.workunit.client.0.trial105.stderr:HEAD is now at 325850157d3 Merge branch 'issue73891' of https://github.com/bill-scales/ceph into wip-bharath18-testing-2026-02-13-0856 2026-02-17T04:44:34.846 DEBUG:teuthology.orchestra.run.trial105:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T04:44:34.895 DEBUG:teuthology.orchestra.run.trial105:> set -ex 2026-02-17T04:44:34.895 DEBUG:teuthology.orchestra.run.trial105:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T04:44:34.941 INFO:tasks.workunit:Running workunits matching misc on client.0... 2026-02-17T04:44:34.942 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2026-02-17T04:44:34.942 DEBUG:teuthology.orchestra.run.trial105:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2026-02-17T04:44:34.995 INFO:tasks.workunit.client.0.trial105.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T04:44:35.000 INFO:tasks.workunit.client.0.trial105.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T04:44:35.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:44:35.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:44:35.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T04:44:35.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/mclock-config 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2026-02-17T04:44:35.001 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2026-02-17T04:44:35.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none ' 2026-02-17T04:44:35.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-17T04:44:35.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2026-02-17T04:44:35.003 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:TEST_profile_builtin_to_custom 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:TEST_profile_custom_to_builtin 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:TEST_recovery_limit_adjustment_mclock' 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:35.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:35.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:35.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:35.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:35.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:35.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:35.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:35.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:35.006 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:35.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:35.007 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:35.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:35.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:35.008 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:35.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:35.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:35.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:35.010 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:35.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:35.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:35.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:35.013 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:35.014 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.014 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:35.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:35.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:35.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:44:35.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:44:35.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.016 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:44:35.017 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:44:35.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:35.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:35.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:35.022 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:35.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:35.023 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:35.023 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:35.023 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:35.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:35.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:35.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:35.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:35.026 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:35.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:35.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:35.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:35.029 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:35.030 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.030 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:35.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:35.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:35.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:44:35.032 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:44:35.032 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:239: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T04:44:35.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:44:35.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:44:35.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:44:35.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T04:44:35.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T04:44:35.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:44:35.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:35.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:35.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:35.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:35.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:44:35.093 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:44:35.095 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:44:35.096 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:35.097 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.097 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.097 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:44:35.097 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:44:35.097 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get fsid 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:44:35.150 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:44:35.151 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:44:35.152 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:35.152 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.152 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.152 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:44:35.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:44:35.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get mon_host 2026-02-17T04:44:35.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:240: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-17T04:44:35.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T04:44:35.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:44:35.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:44:35.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:44:35.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T04:44:35.206 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:44:35.313 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:44:35.313 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:35.313 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:35.313 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:35.313 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.313 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:35.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:44:35.314 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:44:35.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:242: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T04:44:35.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T04:44:35.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:44:35.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:44:35.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:44:35.337 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T04:44:35.337 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T04:44:35.337 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:44:35.337 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:44:35.337 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:35.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T04:44:35.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T04:44:35.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:44:35.340 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c56a3c8c-95fb-48ba-a8f4-41400f27faf5 2026-02-17T04:44:35.340 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c56a3c8c-95fb-48ba-a8f4-41400f27faf5' 2026-02-17T04:44:35.340 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 c56a3c8c-95fb-48ba-a8f4-41400f27faf5 2026-02-17T04:44:35.341 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:44:35.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAz8pNpduWYFBAARV7Iwg6qL2s0+gLIkfthow== 2026-02-17T04:44:35.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAz8pNpduWYFBAARV7Iwg6qL2s0+gLIkfthow=="}' 2026-02-17T04:44:35.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c56a3c8c-95fb-48ba-a8f4-41400f27faf5 -i td/mclock-config/0/new.json 2026-02-17T04:44:35.462 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:35.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T04:44:35.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQAz8pNpduWYFBAARV7Iwg6qL2s0+gLIkfthow== --osd-uuid c56a3c8c-95fb-48ba-a8f4-41400f27faf5 2026-02-17T04:44:35.487 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:35.478+0000 7ff4a3163a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:35.489 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:35.478+0000 7ff4a3163a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:35.490 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:35.482+0000 7ff4a3163a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:35.490 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:35.482+0000 7ff4a3163a40 -1 bdev(0x56317b6e0400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:44:35.490 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:35.482+0000 7ff4a3163a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T04:44:36.259 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T04:44:36.259 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:44:36.260 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:44:36.260 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:44:36.261 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:44:36.378 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:44:36.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:44:36.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:44:36.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T04:44:36.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:44:36.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:44:36.393 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:36.386+0000 7fc9174e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:36.395 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:36.386+0000 7fc9174e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:36.396 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:36.386+0000 7fc9174e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:36.501 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:44:36.501 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:44:36.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:44:36.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:44:36.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:44:36.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:36.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:44:36.503 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:36.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:36.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:36.611 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:36.811 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:36.802+0000 7fc9174e4a40 -1 Falling back to public interface 2026-02-17T04:44:37.420 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:37.410+0000 7fc9174e4a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:44:37.613 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:37.613 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:37.613 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:44:37.614 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:44:37.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:37.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:37.721 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:38.722 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:38.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:38.723 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:44:38.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:44:38.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:38.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:38.933 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3364950980,v1:127.0.0.1:6801/3364950980] [v2:127.0.0.1:6802/3364950980,v1:127.0.0.1:6803/3364950980] exists,up c56a3c8c-95fb-48ba-a8f4-41400f27faf5 2026-02-17T04:44:38.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:44:38.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:44:38.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:44:38.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T04:44:38.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:38.935 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:38.935 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:38.935 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:38.935 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:38.936 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:38.936 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:38.936 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:38.936 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:38.936 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:44:38.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: local backfills=1 2026-02-17T04:44:38.995 INFO:tasks.workunit.client.0.trial105.stdout:osd_max_backfills: 1 2026-02-17T04:44:38.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: echo 'osd_max_backfills: 1' 2026-02-17T04:44:38.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-17T04:44:39.210 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-17T04:44:41.212 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T04:44:41.212 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:41.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:41.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:41.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:41.213 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:41.213 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:41.213 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:41.214 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:41.214 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:41.214 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:44:41.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: local max_backfills=1 2026-02-17T04:44:41.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-17T04:44:41.276 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-17T04:44:41.276 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:41.277 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:41.277 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:41.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T04:44:41.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-17T04:44:41.336 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:260: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-17T04:44:41.337 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-17T04:44:41.337 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:41.337 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:41.338 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:41.338 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:41.338 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:41.339 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:41.339 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:41.339 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:41.339 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:41.339 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T04:44:41.397 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-17T04:44:41.397 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:263: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-17T04:44:41.397 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:267: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-17T04:44:41.611 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:268: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-17T04:44:41.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:269: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-17T04:44:43.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T04:44:43.839 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:43.840 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:43.840 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:43.840 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:43.840 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:43.840 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:43.841 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:43.841 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:43.841 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:43.841 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:44:43.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T04:44:43.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:272: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T04:44:43.902 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-17T04:44:43.902 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:43.903 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:43.903 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:43.904 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:43.904 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:43.904 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:43.904 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:43.904 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:43.904 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:43.905 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T04:44:43.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T04:44:43.964 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:277: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T04:44:43.965 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-17T04:44:43.965 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:43.965 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:43.965 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:43.965 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:43.966 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:43.966 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:43.966 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:43.966 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:43.966 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:43.966 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T04:44:44.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:280: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:284: TEST_backfill_limit_adjustment_mclock: kill_daemons td/mclock-config TERM osd 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:44.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:44.334 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:44.334 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:285: TEST_backfill_limit_adjustment_mclock: ceph osd down 0 2026-02-17T04:44:44.550 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 is already down. 2026-02-17T04:44:44.559 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:286: TEST_backfill_limit_adjustment_mclock: wait_for_osd down 0 2026-02-17T04:44:44.560 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-17T04:44:44.560 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:44:44.560 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:44:44.560 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:44:44.560 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:44.560 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:44.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:44:44.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:44.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-17T04:44:44.784 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3364950980,v1:127.0.0.1:6801/3364950980] [v2:127.0.0.1:6802/3364950980,v1:127.0.0.1:6803/3364950980] exists c56a3c8c-95fb-48ba-a8f4-41400f27faf5 2026-02-17T04:44:44.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:44:44.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:44:44.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:44:44.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:287: TEST_backfill_limit_adjustment_mclock: activate_osd td/mclock-config 0 --osd-op-queue=mclock_scheduler 2026-02-17T04:44:44.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/mclock-config 2026-02-17T04:44:44.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/mclock-config/0 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T04:44:44.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T04:44:44.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=--osd-op-queue=mclock_scheduler 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/mclock-config/0 2026-02-17T04:44:44.791 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:44:44.792 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T04:44:44.792 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler 2026-02-17T04:44:44.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/mclock-config/0/whoami 2026-02-17T04:44:44.793 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T04:44:44.793 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T04:44:44.793 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T04:44:44.793 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T04:44:44.806 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:44.797+0000 7f82daca1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:44.807 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:44.797+0000 7f82daca1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:44.808 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:44.797+0000 7f82daca1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:45.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T04:44:45.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:44:45.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:44:45.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:44:45.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:44:45.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:45.010 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:45.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:44:45.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:45.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:45.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:45.263 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:45.253+0000 7f82daca1a40 -1 Falling back to public interface 2026-02-17T04:44:45.879 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:45.869+0000 7f82daca1a40 -1 osd.0 6 log_to_monitors true 2026-02-17T04:44:46.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:46.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:46.222 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:44:46.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:44:46.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:46.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:46.435 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/3653998727,v1:127.0.0.1:6801/3653998727] [v2:127.0.0.1:6802/3653998727,v1:127.0.0.1:6803/3653998727] exists,up c56a3c8c-95fb-48ba-a8f4-41400f27faf5 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:46.437 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:46.438 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:46.438 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:46.438 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.438 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.438 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:46.439 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:46.439 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:44:46.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T04:44:46.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:291: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T04:44:46.499 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-17T04:44:46.499 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:46.499 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:46.499 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:46.500 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T04:44:46.557 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T04:44:46.557 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:296: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.559 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:46.560 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:44:46.560 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T04:44:46.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:299: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:301: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:46.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:46.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:46.725 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:46.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:46.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:46.727 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:46.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:46.728 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:46.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:46.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:46.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:46.730 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:46.730 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:46.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:46.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:46.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:46.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:46.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:46.740 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:46.740 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:46.741 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:46.741 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:46.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:46.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:46.743 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:46.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:46.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:46.745 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:46.745 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:46.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:46.747 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:46.747 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:46.749 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:46.749 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.749 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:46.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:46.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:46.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:46.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:46.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:46.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:46.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:46.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:46.753 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:46.754 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:46.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:46.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:46.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:46.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:46.757 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:46.757 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:46.757 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:46.757 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:46.758 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:46.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:46.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:46.761 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:46.761 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.761 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:46.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:46.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:46.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:44:46.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:44:46.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:44:46.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:41: TEST_profile_builtin_to_custom: run_mon td/mclock-config a 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T04:44:46.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:44:46.767 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:44:46.767 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:44:46.767 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T04:44:46.767 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T04:44:46.786 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:44:46.786 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:46.787 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:46.787 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:46.787 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.787 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.787 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:46.787 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:44:46.815 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:44:46.816 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:44:46.816 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:44:46.816 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:44:46.817 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:44:46.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get fsid 2026-02-17T04:44:46.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:44:46.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:44:46.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:44:46.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:44:46.875 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:44:46.875 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:44:46.875 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:44:46.875 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:44:46.876 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:46.876 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:46.876 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:46.876 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:44:46.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:44:46.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get mon_host 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T04:44:46.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:44:47.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:44:47.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:47.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:47.043 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:47.043 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:47.043 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:47.043 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:47.043 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:44:47.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:44:47.064 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T04:44:47.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:47.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T04:44:47.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:44:47.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0bfbe309-a24a-42b5-a463-f551f10dbaf8 2026-02-17T04:44:47.068 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 0bfbe309-a24a-42b5-a463-f551f10dbaf8 2026-02-17T04:44:47.069 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0bfbe309-a24a-42b5-a463-f551f10dbaf8' 2026-02-17T04:44:47.069 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:44:47.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA/8pNpZjU8BBAAkp4TF8AUwPxz/f4Tk87Ivw== 2026-02-17T04:44:47.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA/8pNpZjU8BBAAkp4TF8AUwPxz/f4Tk87Ivw=="}' 2026-02-17T04:44:47.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0bfbe309-a24a-42b5-a463-f551f10dbaf8 -i td/mclock-config/0/new.json 2026-02-17T04:44:47.187 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:47.195 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T04:44:47.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQA/8pNpZjU8BBAAkp4TF8AUwPxz/f4Tk87Ivw== --osd-uuid 0bfbe309-a24a-42b5-a463-f551f10dbaf8 2026-02-17T04:44:47.211 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:47.201+0000 7fc327824a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:47.213 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:47.201+0000 7fc327824a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:47.213 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:47.205+0000 7fc327824a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:47.214 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:47.205+0000 7fc327824a40 -1 bdev(0x560886f1e400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:44:47.214 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:47.205+0000 7fc327824a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T04:44:47.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T04:44:47.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:44:47.997 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:44:47.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:44:47.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:44:48.119 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:44:48.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:44:48.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T04:44:48.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:44:48.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:44:48.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:44:48.145 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:48.133+0000 7f4d6563da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:48.147 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:48.137+0000 7f4d6563da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:48.148 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:48.137+0000 7f4d6563da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:48.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:48.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:48.663 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:48.653+0000 7f4d6563da40 -1 Falling back to public interface 2026-02-17T04:44:49.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:49.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:49.344 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:44:49.344 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:44:49.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:49.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:49.550 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:49.650 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:49.641+0000 7f4d6563da40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:44:50.551 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:50.551 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:50.552 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:44:50.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:44:50.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:50.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:50.770 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1938362883,v1:127.0.0.1:6801/1938362883] [v2:127.0.0.1:6802/1938362883,v1:127.0.0.1:6803/1938362883] exists,up 0bfbe309-a24a-42b5-a463-f551f10dbaf8 2026-02-17T04:44:50.770 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:44:50.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:44:50.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:44:50.771 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2026-02-17T04:44:50.984 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: local mclock_profile=balanced 2026-02-17T04:44:50.985 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: test balanced = balanced 2026-02-17T04:44:50.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-17T04:44:50.985 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T04:44:50.986 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:50.986 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:50.986 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:50.986 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:50.986 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:50.986 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:50.987 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T04:44:50.987 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T04:44:51.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: mclock_profile='"high_recovery_ops"' 2026-02-17T04:44:51.047 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: eval echo '"high_recovery_ops"' 2026-02-17T04:44:51.047 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: echo high_recovery_ops 2026-02-17T04:44:51.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: mclock_profile=high_recovery_ops 2026-02-17T04:44:51.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_recovery_ops = high_recovery_ops 2026-02-17T04:44:51.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:57: TEST_profile_builtin_to_custom: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-17T04:44:51.159 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:44:51.159 INFO:tasks.workunit.client.0.trial105.stdout: "success": "" 2026-02-17T04:44:51.159 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.167 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:51.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T04:44:51.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T04:44:51.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: mclock_profile='"custom"' 2026-02-17T04:44:51.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: eval echo '"custom"' 2026-02-17T04:44:51.229 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: echo custom 2026-02-17T04:44:51.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: mclock_profile=custom 2026-02-17T04:44:51.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: test custom = custom 2026-02-17T04:44:51.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-17T04:44:51.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: bc 2026-02-17T04:44:51.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T04:44:51.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:51.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:51.233 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:51.233 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.233 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.234 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:51.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T04:44:51.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T04:44:51.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: local client_res=0.000000 2026-02-17T04:44:51.291 INFO:tasks.workunit.client.0.trial105.stdout:client_res = 0.000000 2026-02-17T04:44:51.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:69: TEST_profile_builtin_to_custom: echo 'client_res = 0.000000' 2026-02-17T04:44:51.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: echo '0.000000 + 0.1' 2026-02-17T04:44:51.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc -l 2026-02-17T04:44:51.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res_new=.100000 2026-02-17T04:44:51.293 INFO:tasks.workunit.client.0.trial105.stdout:client_res_new = .100000 2026-02-17T04:44:51.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res_new = .100000' 2026-02-17T04:44:51.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: ceph config set osd.0 osd_mclock_scheduler_client_res .100000 2026-02-17T04:44:51.513 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T04:44:51.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: local res=0.100000 2026-02-17T04:44:51.729 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-17T04:44:51.730 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: bc -l 2026-02-17T04:44:51.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-17T04:44:51.731 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-17T04:44:51.732 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: bc 2026-02-17T04:44:51.732 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T04:44:51.732 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:51.732 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:51.733 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:51.733 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.733 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.733 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:51.733 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T04:44:51.733 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T04:44:51.793 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: res=0.100000 2026-02-17T04:44:51.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-17T04:44:51.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc -l 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:51.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:51.903 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:51.904 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:51.904 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:51.905 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:51.906 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:51.906 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:51.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:51.908 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:51.908 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.909 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:51.910 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:51.910 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:51.913 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:51.913 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.913 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.913 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:51.915 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:51.915 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:51.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:51.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:51.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:51.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:51.919 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:51.920 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:51.920 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:51.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:51.921 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.922 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:51.922 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:51.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.923 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:51.924 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:51.925 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:51.926 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:51.926 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.926 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.926 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:51.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:51.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:51.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T04:44:51.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T04:44:51.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:44:51.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:51.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:51.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:51.930 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:51.931 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:51.932 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:51.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:51.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:51.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:51.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:51.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:51.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.936 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:51.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:51.938 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:51.939 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:51.939 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.939 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:51.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:51.941 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:51.941 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:44:51.942 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:44:51.942 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.942 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.942 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:44:51.944 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:44:51.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:44:51.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:44:51.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:95: TEST_profile_custom_to_builtin: setup td/mclock-config 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:44:51.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:44:51.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:44:51.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:44:51.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:44:51.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:44:51.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:44:51.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:44:51.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:44:51.951 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.951 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:44:51.952 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:44:51.952 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:44:51.953 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:44:51.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:44:51.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:44:51.955 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:44:51.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.956 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:44:51.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:44:51.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:44:51.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:44:51.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:44:51.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:44:51.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: run_mon td/mclock-config a 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T04:44:51.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:51.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:51.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:44:52.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:44:52.010 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:44:52.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:44:52.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:44:52.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:44:52.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:44:52.011 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:44:52.011 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:44:52.012 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:44:52.012 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:52.012 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:52.012 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:52.012 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:44:52.013 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:44:52.013 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get fsid 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:44:52.067 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:44:52.068 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:52.068 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:52.068 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:52.068 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:44:52.069 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:44:52.069 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get mon_host 2026-02-17T04:44:52.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2026-02-17T04:44:52.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T04:44:52.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:44:52.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:44:52.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:44:52.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T04:44:52.123 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:44:52.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:44:52.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:52.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:52.232 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:52.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:52.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:52.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:52.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:44:52.234 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:44:52.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T04:44:52.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T04:44:52.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:44:52.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:44:52.254 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T04:44:52.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T04:44:52.256 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:44:52.257 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=09d068f0-82b1-4b6b-a3cb-03c37c4b05c5 2026-02-17T04:44:52.257 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 09d068f0-82b1-4b6b-a3cb-03c37c4b05c5' 2026-02-17T04:44:52.257 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 09d068f0-82b1-4b6b-a3cb-03c37c4b05c5 2026-02-17T04:44:52.257 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:44:52.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBE8pNp/zOODxAAIvIrCW+bAxteqWPU7k4cSw== 2026-02-17T04:44:52.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBE8pNp/zOODxAAIvIrCW+bAxteqWPU7k4cSw=="}' 2026-02-17T04:44:52.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 09d068f0-82b1-4b6b-a3cb-03c37c4b05c5 -i td/mclock-config/0/new.json 2026-02-17T04:44:52.378 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:52.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T04:44:52.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQBE8pNp/zOODxAAIvIrCW+bAxteqWPU7k4cSw== --osd-uuid 09d068f0-82b1-4b6b-a3cb-03c37c4b05c5 2026-02-17T04:44:52.403 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:52.393+0000 7fa0b9afca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:52.404 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:52.393+0000 7fa0b9afca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:52.405 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:52.393+0000 7fa0b9afca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:52.405 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:52.397+0000 7fa0b9afca40 -1 bdev(0x560e17b84400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:44:52.406 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:52.397+0000 7fa0b9afca40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T04:44:53.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T04:44:53.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:44:53.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:44:53.187 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:44:53.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:44:53.305 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:44:53.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:44:53.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T04:44:53.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:44:53.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:44:53.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:44:53.331 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:53.321+0000 7f957ca8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:53.333 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:53.325+0000 7f957ca8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:53.335 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:53.325+0000 7f957ca8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:44:53.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:53.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:53.783 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:53.773+0000 7f957ca8ca40 -1 Falling back to public interface 2026-02-17T04:44:54.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:54.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:54.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:44:54.529 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:44:54.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:54.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:54.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:44:54.815 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:44:54.805+0000 7f957ca8ca40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:44:55.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:44:55.639 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:44:55.639 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:44:55.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:44:55.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:44:55.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:44:55.851 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1322847055,v1:127.0.0.1:6801/1322847055] [v2:127.0.0.1:6802/1322847055,v1:127.0.0.1:6803/1322847055] exists,up 09d068f0-82b1-4b6b-a3cb-03c37c4b05c5 2026-02-17T04:44:55.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:44:55.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:44:55.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:44:55.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: local def_mclock_profile 2026-02-17T04:44:55.853 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2026-02-17T04:44:56.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: def_mclock_profile=balanced 2026-02-17T04:44:56.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:103: TEST_profile_custom_to_builtin: test balanced = balanced 2026-02-17T04:44:56.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-17T04:44:56.068 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:44:56.068 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:56.068 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:56.068 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:56.068 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:56.068 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:56.069 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:56.069 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:44:56.069 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T04:44:56.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: local 'orig_mclock_profile="high_recovery_ops"' 2026-02-17T04:44:56.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: eval echo '"high_recovery_ops"' 2026-02-17T04:44:56.130 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: echo high_recovery_ops 2026-02-17T04:44:56.130 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: orig_mclock_profile=high_recovery_ops 2026-02-17T04:44:56.130 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:110: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-17T04:44:56.130 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:113: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-17T04:44:56.218 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:44:56.218 INFO:tasks.workunit.client.0.trial105.stdout: "success": "" 2026-02-17T04:44:56.218 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:56.229 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:56.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:56.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:44:56.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T04:44:56.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: local 'mclock_profile="custom"' 2026-02-17T04:44:56.288 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: eval echo '"custom"' 2026-02-17T04:44:56.288 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: echo custom 2026-02-17T04:44:56.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: mclock_profile=custom 2026-02-17T04:44:56.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:120: TEST_profile_custom_to_builtin: test custom = custom 2026-02-17T04:44:56.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: local client_res 2026-02-17T04:44:56.289 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T04:44:56.289 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: bc 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:56.290 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:56.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:44:56.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T04:44:56.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: client_res=0.000000 2026-02-17T04:44:56.351 INFO:tasks.workunit.client.0.trial105.stdout:Original client_res for osd.0 = 0.000000 2026-02-17T04:44:56.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 0.000000' 2026-02-17T04:44:56.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: echo '0.000000 + 0.1' 2026-02-17T04:44:56.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:44:56.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: local client_res_new=.100000 2026-02-17T04:44:56.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:131: TEST_profile_custom_to_builtin: echo 'client_res_new = .100000' 2026-02-17T04:44:56.353 INFO:tasks.workunit.client.0.trial105.stdout:client_res_new = .100000 2026-02-17T04:44:56.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res .100000 2026-02-17T04:44:56.571 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T04:44:56.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-17T04:44:56.780 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T04:44:56.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:44:56.782 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T04:44:56.783 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T04:44:56.783 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: bc 2026-02-17T04:44:56.783 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:44:56.784 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:56.784 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:56.785 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:56.785 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:56.785 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:56.785 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:56.785 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:44:56.785 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T04:44:56.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: res=0.100000 2026-02-17T04:44:56.846 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T04:44:56.846 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:44:56.848 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T04:44:56.848 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:154: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile high_recovery_ops 2026-02-17T04:44:56.924 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:44:56.924 INFO:tasks.workunit.client.0.trial105.stdout: "success": "" 2026-02-17T04:44:56.925 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:44:56.936 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-17T04:44:56.936 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:44:56.936 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:56.936 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:56.937 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:56.937 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:56.937 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:56.937 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:56.937 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:44:56.937 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T04:44:56.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: eval 'mclock_profile="high_recovery_ops"' 2026-02-17T04:44:56.995 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: mclock_profile=high_recovery_ops 2026-02-17T04:44:56.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-17T04:44:56.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T04:44:57.203 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-17T04:44:57.204 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T04:44:57.204 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:44:57.206 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T04:44:57.207 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T04:44:57.207 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: bc 2026-02-17T04:44:57.207 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:44:57.207 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:44:57.208 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T04:44:57.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: res=0.100000 2026-02-17T04:44:57.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T04:44:57.270 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:44:57.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T04:44:57.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2026-02-17T04:44:57.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:180: TEST_profile_custom_to_builtin: sleep 5 2026-02-17T04:45:02.490 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T04:45:02.702 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: res=0.000000 2026-02-17T04:45:02.703 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: echo '0.000000 != 0.0' 2026-02-17T04:45:02.703 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:45:02.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T04:45:02.706 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T04:45:02.706 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: bc 2026-02-17T04:45:02.706 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T04:45:02.706 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:45:02.706 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:45:02.707 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:02.707 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.707 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.707 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:45:02.707 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T04:45:02.707 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T04:45:02.768 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: res=0.000000 2026-02-17T04:45:02.769 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: echo '0.000000 != 0.000000' 2026-02-17T04:45:02.769 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: bc -l 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_profile_custom_to_builtin: teardown td/mclock-config 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:02.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:02.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:02.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:02.878 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:02.879 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:02.879 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:02.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:02.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:02.880 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:02.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:02.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:02.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:02.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:02.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:02.887 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:02.887 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.887 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:02.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:02.891 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:02.892 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:02.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:02.893 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:02.893 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:02.894 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:02.895 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:02.895 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:02.895 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.896 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:02.896 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:02.896 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.897 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:02.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:02.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:02.899 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:02.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:45:02.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:02.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:02.904 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:02.904 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:02.905 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:02.905 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:02.906 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:02.906 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:02.906 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:02.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.908 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:02.908 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:02.908 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.909 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:02.910 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:02.910 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:02.912 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:02.912 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.912 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.912 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:02.913 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:02.913 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:02.913 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:45:02.914 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:45:02.915 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.915 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.915 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:45:02.916 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:45:02.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:45:02.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:02.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:02.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:02.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:02.920 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:02.920 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:02.920 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:02.920 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:02.921 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:02.921 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:02.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:02.923 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:02.923 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:02.924 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:02.925 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.925 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:02.925 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:02.925 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:02.926 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:02.927 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:02.927 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:02.928 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:02.929 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.929 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:02.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:02.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:02.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T04:45:02.931 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:45:02.931 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.931 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.931 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11382 2026-02-17T04:45:02.933 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:45:02.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:45:02.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:45:02.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T04:45:02.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:205: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-17T04:45:02.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T04:45:02.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:45:02.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:45:02.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:45:02.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T04:45:02.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T04:45:02.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:45:02.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:02.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:02.957 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:02.957 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.957 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.957 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:45:02.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:45:02.985 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:45:02.986 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:45:02.987 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:45:02.987 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:45:02.987 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:45:02.987 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:45:02.988 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:45:02.988 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:45:02.988 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:45:02.988 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:02.988 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:02.989 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:02.989 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:45:02.989 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:45:02.989 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get fsid 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:45:03.061 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:45:03.062 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:03.062 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:03.062 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:03.062 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-mon.a.asok 2026-02-17T04:45:03.062 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:45:03.062 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11382/ceph-mon.a.asok config get mon_host 2026-02-17T04:45:03.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:206: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-17T04:45:03.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T04:45:03.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:45:03.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:45:03.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:45:03.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T04:45:03.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:45:03.245 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:45:03.246 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:03.246 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:03.246 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:03.246 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:03.246 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:03.246 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:45:03.247 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:45:03.248 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:208: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:03.267 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:03.268 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:03.268 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:03.268 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:03.268 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:45:03.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' 2026-02-17T04:45:03.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=94e7d9a8-b117-47b0-9359-7eedfdfa3f77 2026-02-17T04:45:03.269 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 94e7d9a8-b117-47b0-9359-7eedfdfa3f77 2026-02-17T04:45:03.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 94e7d9a8-b117-47b0-9359-7eedfdfa3f77' 2026-02-17T04:45:03.270 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:45:03.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBP8pNp27JnEBAAiDqF6Bas/12vItF0HHSDMg== 2026-02-17T04:45:03.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBP8pNp27JnEBAAiDqF6Bas/12vItF0HHSDMg=="}' 2026-02-17T04:45:03.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 94e7d9a8-b117-47b0-9359-7eedfdfa3f77 -i td/mclock-config/0/new.json 2026-02-17T04:45:03.393 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:45:03.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T04:45:03.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQBP8pNp27JnEBAAiDqF6Bas/12vItF0HHSDMg== --osd-uuid 94e7d9a8-b117-47b0-9359-7eedfdfa3f77 2026-02-17T04:45:03.418 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:03.409+0000 7f62e24a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:03.419 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:03.409+0000 7f62e24a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:03.421 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:03.409+0000 7f62e24a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:03.421 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:03.413+0000 7f62e24a8a40 -1 bdev(0x55630de36400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:45:03.421 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:03.413+0000 7f62e24a8a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T04:45:04.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T04:45:04.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:45:04.169 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:45:04.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:45:04.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:45:04.290 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:45:04.291 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:45:04.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=85ed0fe1-258e-46a3-a366-86bde854a539 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.11382/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T04:45:04.292 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:45:04.292 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:45:04.292 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:45:04.313 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:04.301+0000 7ff90220ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:04.315 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:04.305+0000 7ff90220ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:04.316 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:04.305+0000 7ff90220ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:04.401 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:45:04.402 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:45:04.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:04.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:45:04.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:04.767 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:04.757+0000 7ff90220ba40 -1 Falling back to public interface 2026-02-17T04:45:05.445 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:05.433+0000 7ff90220ba40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:45:05.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:05.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:05.510 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:45:05.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:45:05.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:05.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:45:05.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:06.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:06.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:06.620 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:45:06.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:45:06.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:06.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:45:06.836 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3049577258,v1:127.0.0.1:6801/3049577258] [v2:127.0.0.1:6802/3049577258,v1:127.0.0.1:6803/3049577258] exists,up 94e7d9a8-b117-47b0-9359-7eedfdfa3f77 2026-02-17T04:45:06.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:45:06.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:45:06.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:45:06.839 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-17T04:45:06.897 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2026-02-17T04:45:06.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2026-02-17T04:45:06.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2026-02-17T04:45:06.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-17T04:45:07.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-17T04:45:09.125 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:45:09.125 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:45:09.125 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:45:09.125 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:09.125 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:09.125 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:09.126 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:45:09.126 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:45:09.126 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-17T04:45:09.184 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2026-02-17T04:45:09.184 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2026-02-17T04:45:09.184 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:225: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-17T04:45:09.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:226: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-17T04:45:09.636 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:227: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-17T04:45:11.638 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T04:45:11.639 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:45:11.639 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:45:11.640 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:11.640 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:11.640 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:11.640 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11382/ceph-osd.0.asok 2026-02-17T04:45:11.640 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T04:45:11.640 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.11382/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:11.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:11.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:11.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:11.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:11.807 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:11.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:11.808 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:11.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:11.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:11.810 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:11.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:11.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:11.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.812 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:11.813 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:11.813 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:11.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:11.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:11.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:11.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:11.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:11.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:11.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T04:45:11.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:11.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:11.823 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:11.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:11.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:11.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:11.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:11.825 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:11.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:11.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:11.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:11.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:11.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:11.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:11.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:11.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:11.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/mclock-config 0 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:11.833 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:11.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:11.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:11.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:11.835 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:11.835 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:11.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:11.836 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:11.838 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:11.838 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:11.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:11.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:11.840 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:11.840 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:11.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T04:45:11.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T04:45:11.843 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:11.843 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:11.844 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11382 2026-02-17T04:45:11.844 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11382 2026-02-17T04:45:11.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:11.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:11.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T04:45:11.846 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T04:45:11.846 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T04:45:11.902 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2026-02-17T04:45:11.902 DEBUG:teuthology.orchestra.run.trial105:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2026-02-17T04:45:11.955 INFO:tasks.workunit.client.0.trial105.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T04:45:11.958 INFO:tasks.workunit.client.0.trial105.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T04:45:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:45:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/network-ping 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2026-02-17T04:45:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2026-02-17T04:45:11.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2026-02-17T04:45:11.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none ' 2026-02-17T04:45:11.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-17T04:45:11.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2026-02-17T04:45:11.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2026-02-17T04:45:11.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2026-02-17T04:45:11.961 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2026-02-17T04:45:11.961 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:TEST_network_ping_test2' 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T04:45:11.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:45:11.964 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:45:11.965 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:45:11.965 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:45:11.966 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:45:11.967 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:45:11.967 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:45:11.968 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:45:11.968 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:45:11.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:45:11.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:45:11.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:45:11.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:45:11.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T04:45:11.973 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:45:11.973 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:11.973 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:11.973 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.23392 2026-02-17T04:45:11.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:45:11.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:45:11.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-17T04:45:11.976 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:45:11.976 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:11.976 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:11.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.23392 2026-02-17T04:45:11.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-17T04:45:11.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:45:11.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:45:11.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:45:11.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-17T04:45:11.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-17T04:45:12.004 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:45:12.004 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:12.004 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:12.004 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:12.004 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:12.005 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:12.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:12.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:45:12.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:45:12.036 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:45:12.036 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:45:12.036 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:45:12.036 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:45:12.036 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mon.a.asok 2026-02-17T04:45:12.038 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:45:12.039 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.23392/ceph-mon.a.asok config get fsid 2026-02-17T04:45:12.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:45:12.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:45:12.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:45:12.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:45:12.094 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mon.a.asok 2026-02-17T04:45:12.095 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:45:12.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.23392/ceph-mon.a.asok config get mon_host 2026-02-17T04:45:12.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2026-02-17T04:45:12.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-17T04:45:12.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:45:12.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:45:12.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:45:12.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-17T04:45:12.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:45:12.264 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:45:12.265 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:12.265 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:12.265 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:12.265 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:12.265 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:12.265 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:12.266 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:45:12.266 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-17T04:45:12.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:12.286 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:45:12.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:45:12.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:45:12.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:45:12.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:45:12.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-17T04:45:12.288 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:45:12.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d17b7df6-80e6-4332-8adb-07142b58b9b6 2026-02-17T04:45:12.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d17b7df6-80e6-4332-8adb-07142b58b9b6' 2026-02-17T04:45:12.289 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 d17b7df6-80e6-4332-8adb-07142b58b9b6 2026-02-17T04:45:12.289 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:45:12.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBY8pNpzolvERAAmS4Z8sbHNq4Xle7UzvkrFg== 2026-02-17T04:45:12.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBY8pNpzolvERAAmS4Z8sbHNq4Xle7UzvkrFg=="}' 2026-02-17T04:45:12.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d17b7df6-80e6-4332-8adb-07142b58b9b6 -i td/network-ping/0/new.json 2026-02-17T04:45:12.409 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:45:12.416 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-17T04:45:12.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBY8pNpzolvERAAmS4Z8sbHNq4Xle7UzvkrFg== --osd-uuid d17b7df6-80e6-4332-8adb-07142b58b9b6 2026-02-17T04:45:12.435 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:12.424+0000 7f19457a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:12.436 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:12.424+0000 7f19457a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:12.437 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:12.424+0000 7f19457a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:12.437 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:12.424+0000 7f19457a5a40 -1 bdev(0x558ab0116400 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:45:12.438 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:12.428+0000 7f19457a5a40 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-17T04:45:13.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-17T04:45:13.081 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:45:13.081 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:45:13.082 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:45:13.082 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:45:13.190 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:45:13.190 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:45:13.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:45:13.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:45:13.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:45:13.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:45:13.206 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:13.196+0000 7fbec5df6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:13.207 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:13.196+0000 7fbec5df6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:13.208 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:13.196+0000 7fbec5df6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:13.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:45:13.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:45:13.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:45:13.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:45:13.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:45:13.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:13.304 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:45:13.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:45:13.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:13.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:45:13.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:13.735 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:13.724+0000 7fbec5df6a40 -1 Falling back to public interface 2026-02-17T04:45:14.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:14.416 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:14.416 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:45:14.416 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:45:14.417 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:45:14.420 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:14.545 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:14.744 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:14.732+0000 7fbec5df6a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:45:15.546 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:15.547 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:15.547 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:45:15.548 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:45:15.548 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:15.548 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:45:15.761 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2452723545,v1:127.0.0.1:6801/2452723545] [v2:127.0.0.1:6802/2452723545,v1:127.0.0.1:6803/2452723545] exists,up d17b7df6-80e6-4332-8adb-07142b58b9b6 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:45:15.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T04:45:15.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-17T04:45:15.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:45:15.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1c039e4b-f0d7-4591-859b-39d40fe45f64 2026-02-17T04:45:15.765 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 1c039e4b-f0d7-4591-859b-39d40fe45f64 2026-02-17T04:45:15.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1c039e4b-f0d7-4591-859b-39d40fe45f64' 2026-02-17T04:45:15.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:45:15.777 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBb8pNpyuz6LRAAKb61NAuWeRQDgib1KpmSWQ== 2026-02-17T04:45:15.777 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBb8pNpyuz6LRAAKb61NAuWeRQDgib1KpmSWQ=="}' 2026-02-17T04:45:15.777 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1c039e4b-f0d7-4591-859b-39d40fe45f64 -i td/network-ping/1/new.json 2026-02-17T04:45:16.002 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:45:16.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-17T04:45:16.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBb8pNpyuz6LRAAKb61NAuWeRQDgib1KpmSWQ== --osd-uuid 1c039e4b-f0d7-4591-859b-39d40fe45f64 2026-02-17T04:45:16.029 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.016+0000 7fa18782da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:16.030 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.020+0000 7fa18782da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:16.031 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.020+0000 7fa18782da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:16.032 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.020+0000 7fa18782da40 -1 bdev(0x55a15530a400 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:45:16.032 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.020+0000 7fa18782da40 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-17T04:45:16.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-17T04:45:16.660 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:45:16.661 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:45:16.661 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:45:16.661 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:45:16.956 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:45:16.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:45:16.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:45:16.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:45:16.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:45:16.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:45:16.973 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.960+0000 7fbedc339a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:16.974 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.964+0000 7fbedc339a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:16.975 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:16.964+0000 7fbedc339a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:17.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:45:17.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:45:17.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:45:17.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:45:17.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:45:17.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:17.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:45:17.164 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:45:17.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:17.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:45:17.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:17.387 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:17.376+0000 7fbedc339a40 -1 Falling back to public interface 2026-02-17T04:45:17.995 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:17.984+0000 7fbedc339a40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:45:18.387 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:18.387 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:18.387 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:45:18.387 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:45:18.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:18.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:45:18.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:19.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:19.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:19.600 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:45:19.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:45:19.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:19.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:45:19.819 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3826832731,v1:127.0.0.1:6811/3826832731] [v2:127.0.0.1:6812/3826832731,v1:127.0.0.1:6813/3826832731] exists,up 1c039e4b-f0d7-4591-859b-39d40fe45f64 2026-02-17T04:45:19.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T04:45:19.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:45:19.824 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:45:19.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:45:19.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:45:19.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:45:19.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-17T04:45:19.826 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:45:19.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d792d129-8b22-4fa4-a744-a2a70dcf9af2 2026-02-17T04:45:19.827 INFO:tasks.workunit.client.0.trial105.stdout:add osd2 d792d129-8b22-4fa4-a744-a2a70dcf9af2 2026-02-17T04:45:19.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d792d129-8b22-4fa4-a744-a2a70dcf9af2' 2026-02-17T04:45:19.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:45:19.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBf8pNp5O+FMRAAZV3M+eclOMhHFSk0CfZ1dA== 2026-02-17T04:45:19.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBf8pNp5O+FMRAAZV3M+eclOMhHFSk0CfZ1dA=="}' 2026-02-17T04:45:19.838 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d792d129-8b22-4fa4-a744-a2a70dcf9af2 -i td/network-ping/2/new.json 2026-02-17T04:45:20.055 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:45:20.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-17T04:45:20.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBf8pNp5O+FMRAAZV3M+eclOMhHFSk0CfZ1dA== --osd-uuid d792d129-8b22-4fa4-a744-a2a70dcf9af2 2026-02-17T04:45:20.092 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:20.080+0000 7f540c97ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:20.094 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:20.084+0000 7f540c97ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:20.095 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:20.084+0000 7f540c97ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:20.095 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:20.084+0000 7f540c97ea40 -1 bdev(0x55bb7f4ea400 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-17T04:45:20.096 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:20.084+0000 7f540c97ea40 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-17T04:45:20.709 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-17T04:45:20.709 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:45:20.710 INFO:tasks.workunit.client.0.trial105.stdout:adding osd2 key to auth repository 2026-02-17T04:45:20.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T04:45:20.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:45:21.013 INFO:tasks.workunit.client.0.trial105.stdout:start osd.2 2026-02-17T04:45:21.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T04:45:21.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:45:21.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:45:21.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:45:21.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:45:21.031 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:21.020+0000 7f0f358a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:21.032 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:21.020+0000 7f0f358a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:21.033 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:21.020+0000 7f0f358a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:45:21.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T04:45:21.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:45:21.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T04:45:21.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:45:21.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:45:21.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:21.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:45:21.224 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:45:21.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:21.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:45:21.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:21.491 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:21.480+0000 7f0f358a8a40 -1 Falling back to public interface 2026-02-17T04:45:22.084 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:45:22.072+0000 7f0f358a8a40 -1 osd.2 0 log_to_monitors true 2026-02-17T04:45:22.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:22.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:22.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:45:22.448 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:45:22.448 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:22.448 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:45:22.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:45:23.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:45:23.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:45:23.664 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:45:23.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:45:23.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:45:23.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:45:23.881 INFO:tasks.workunit.client.0.trial105.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/241629649,v1:127.0.0.1:6819/241629649] [v2:127.0.0.1:6820/241629649,v1:127.0.0.1:6821/241629649] exists,up d792d129-8b22-4fa4-a744-a2a70dcf9af2 2026-02-17T04:45:23.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:45:23.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:45:23.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:45:23.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2026-02-17T04:45:28.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2026-02-17T04:45:28.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-17T04:45:29.149 INFO:tasks.workunit.client.0.trial105.stderr:pool 'foo' already exists 2026-02-17T04:45:29.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:45:30.160 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-17T04:45:30.181 INFO:tasks.workunit.client.0.trial105.stdout:hints = 1 2026-02-17T04:45:30.181 INFO:tasks.workunit.client.0.trial105.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-17T04:45:30.181 INFO:tasks.workunit.client.0.trial105.stdout:Object prefix: benchmark_data_trial105_28861 2026-02-17T04:45:30.181 INFO:tasks.workunit.client.0.trial105.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T04:45:30.181 INFO:tasks.workunit.client.0.trial105.stdout: 0 0 0 0 0 0 - 0 2026-02-17T04:45:31.181 INFO:tasks.workunit.client.0.trial105.stdout: 1 15 10414 10399 40.6181 40.6211 0.00110534 0.00153617 2026-02-17T04:45:32.182 INFO:tasks.workunit.client.0.trial105.stdout: 2 16 21155 21139 41.284 41.9531 0.000918764 0.0015118 2026-02-17T04:45:33.182 INFO:tasks.workunit.client.0.trial105.stdout: 3 15 32439 32424 42.2155 44.082 0.000773576 0.00147877 2026-02-17T04:45:34.182 INFO:tasks.workunit.client.0.trial105.stdout: 4 16 43695 43679 42.652 43.9648 0.00147837 0.00146374 2026-02-17T04:45:35.182 INFO:tasks.workunit.client.0.trial105.stdout: 5 16 54200 54184 42.328 41.0352 0.00322322 0.00147496 2026-02-17T04:45:36.182 INFO:tasks.workunit.client.0.trial105.stdout: 6 16 65287 65271 42.4908 43.3086 0.000887376 0.00146937 2026-02-17T04:45:37.182 INFO:tasks.workunit.client.0.trial105.stdout: 7 16 76642 76626 42.7567 44.3555 0.00131196 0.00146028 2026-02-17T04:45:38.182 INFO:tasks.workunit.client.0.trial105.stdout: 8 16 87006 86990 42.4723 40.4844 0.00146285 0.00147007 2026-02-17T04:45:39.182 INFO:tasks.workunit.client.0.trial105.stdout: 9 16 96565 96549 41.9017 37.3398 0.001473 0.00149005 2026-02-17T04:45:40.182 INFO:tasks.workunit.client.0.trial105.stdout: 10 2 105919 105917 41.3706 36.5938 0.00120991 0.00150927 2026-02-17T04:45:41.182 INFO:tasks.workunit.client.0.trial105.stdout:Total time run: 10.0011 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Total writes made: 105919 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Write size: 4096 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Object size: 4096 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Bandwidth (MB/sec): 41.3699 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Stddev Bandwidth: 2.74087 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Max bandwidth (MB/sec): 44.3555 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Min bandwidth (MB/sec): 36.5938 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Average IOPS: 10590 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Stddev IOPS: 701.663 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Max IOPS: 11355 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Min IOPS: 9368 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Average Latency(s): 0.00150929 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Stddev Latency(s): 0.000733892 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Max latency(s): 0.0111399 2026-02-17T04:45:41.183 INFO:tasks.workunit.client.0.trial105.stdout:Min latency(s): 0.000383898 2026-02-17T04:45:41.189 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2026-02-17T04:45:51.190 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2026-02-17T04:45:51.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:45:51.191 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:45:51.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:45:51.845 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:45:51.845 INFO:tasks.workunit.client.0.trial105.stderr:2' 2026-02-17T04:45:51.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:45:51.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:45:51.845 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:45:52.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-17T04:45:52.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-17T04:45:52.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T04:45:52.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:45:52.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:45:52.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705673 2026-02-17T04:45:52.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705673 2026-02-17T04:45:52.435 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673' 2026-02-17T04:45:52.435 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:45:52.435 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:45:52.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-17T04:45:52.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-17T04:45:52.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856' 2026-02-17T04:45:52.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:45:52.580 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-17T04:45:52.580 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:45:52.581 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:45:52.581 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-17T04:45:52.582 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:45:52.583 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-17T04:45:52.583 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836490 2026-02-17T04:45:52.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T04:45:52.584 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:45:53.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836490 2026-02-17T04:45:53.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:45:54.060 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:45:54.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:45:54.277 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-17T04:45:54.277 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:45:54.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705673 2026-02-17T04:45:54.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:45:54.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:45:54.280 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705673 2026-02-17T04:45:54.280 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:45:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705673 2026-02-17T04:45:54.281 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705673 2026-02-17T04:45:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-17T04:45:54.282 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:45:54.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-17T04:45:54.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:45:54.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574856 2026-02-17T04:45:54.504 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:45:54.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:45:54.506 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574856 2026-02-17T04:45:54.506 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:45:54.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-17T04:45:54.507 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574856 2026-02-17T04:45:54.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574856' 2026-02-17T04:45:54.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:45:54.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574856 2026-02-17T04:45:54.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:54.727 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:45:54.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:45:54.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 2026-02-17T04:45:54.782 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:45:54.782 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 1000, 2026-02-17T04:45:54.782 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:45:54.782 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:45:54.789 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:54.789 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:45:54.798 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:45:54.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:54.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2026-02-17T04:45:54.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T04:45:54.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:45:54.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:45:54.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:45:54.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:45:54.811 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:54.812 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:54.812 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:54.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:45:54.812 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:45:54.812 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 2026-02-17T04:45:54.859 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:45:54.859 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 1000, 2026-02-17T04:45:54.859 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:45:54.859 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:45:54.866 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:54.866 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:45:54.874 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:45:54.875 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:54.875 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2026-02-17T04:45:54.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T04:45:54.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:45:54.886 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:45:54.886 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:45:54.886 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:45:54.887 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:54.887 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:54.887 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:54.887 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:45:54.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:45:54.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 0 2026-02-17T04:45:54.940 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 0, 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [ 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:52 2026", 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.175, 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.195, 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.228 2026-02-17T04:45:54.941 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.224, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.352, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.468 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.224, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.352, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.468 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.216 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:54.942 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:52 2026", 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.177, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.190, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.225 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.199, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.351, 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.501 2026-02-17T04:45:54.943 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.199, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.351, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.501 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.254 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:50 2026", 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:54.944 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.151, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.199, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.220 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.181, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.479, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.479 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.181, 2026-02-17T04:45:54.945 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.479, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.479 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.231 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:50 2026", 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.162, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.188, 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.172 2026-02-17T04:45:54.946 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.179, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.388, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.388 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.179, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.388, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.388 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.202 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: } 2026-02-17T04:45:54.947 INFO:tasks.workunit.client.0.trial105.stdout: ] 2026-02-17T04:45:54.948 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:45:54.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:54.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:45:54.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2026-02-17T04:45:54.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:54.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2026-02-17T04:45:54.968 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:45:54.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:45:54.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:45:54.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:45:54.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:45:54.970 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:45:54.970 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:45:54.970 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:45:54.970 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:45:54.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:45:54.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 0 2026-02-17T04:45:55.018 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 0, 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [ 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:47 2026", 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:45:55.019 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.151, 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.209, 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.233 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.134, 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.134, 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.134 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.020 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.179, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.468, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.468 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.190 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:47 2026", 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:45:55.021 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.173, 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.204, 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.230 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.142, 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.114, 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.114 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.022 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.203, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.501, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.501 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.178 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:50 2026", 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.023 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.151, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.199, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.220 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.127, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.114, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.114 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.181, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.479, 2026-02-17T04:45:55.024 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.479 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.165 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:53 2026", 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.025 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.192, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.219, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.185 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.157, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.108, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.108 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.242, 2026-02-17T04:45:55.026 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.633, 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.633 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.196 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:53 2026", 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:55.027 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.194, 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.192, 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.189 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.149, 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.133, 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.133 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.028 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.255, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.352, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.352 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.238 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:53 2026", 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:55.029 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.165, 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.178, 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.192 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.140, 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.140, 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.140 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.030 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.203, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.292, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.292 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.203 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:50 2026", 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:45:55.031 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.162, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.188, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.172 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.144, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.110, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.110 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.179, 2026-02-17T04:45:55.032 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.388, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.388 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.208 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:54 2026", 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.162, 2026-02-17T04:45:55.033 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.163, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.187 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.122, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.118, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.111 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.207, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.288, 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.476 2026-02-17T04:45:55.034 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.184 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:54 2026", 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.182, 2026-02-17T04:45:55.035 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.186, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.175 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.135, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.107, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.107 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.222, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.331, 2026-02-17T04:45:55.036 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.331 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.215 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:53 2026", 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.037 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.130, 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.174, 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.170 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.097, 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.097, 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.097 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.038 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.181, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.321, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.321 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.181 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:53 2026", 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:45:55.039 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.152, 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.171, 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.152 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.113, 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.113, 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.113 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.040 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.214, 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.259, 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.259 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.214 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:45:53 2026", 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:45:55.041 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.137, 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.142, 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.154 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.121, 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.116, 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.106 2026-02-17T04:45:55.042 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.158, 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.208, 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.241 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.158 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: } 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout: ] 2026-02-17T04:45:55.043 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:45:55.044 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:55.044 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:45:55.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2026-02-17T04:45:55.045 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:45:55.045 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2026-02-17T04:45:55.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:45:55.046 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2026-02-17T04:46:15.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2026-02-17T04:46:15.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:46:15.048 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:46:15.266 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:46:15.266 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:46:15.266 INFO:tasks.workunit.client.0.trial105.stderr:2' 2026-02-17T04:46:15.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:46:15.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:46:15.267 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:46:15.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-17T04:46:15.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-17T04:46:15.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496' 2026-02-17T04:46:15.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:46:15.346 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:46:15.425 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705680 2026-02-17T04:46:15.425 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705680 2026-02-17T04:46:15.425 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496 1-38654705680' 2026-02-17T04:46:15.425 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:46:15.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:46:15.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-17T04:46:15.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-17T04:46:15.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496 1-38654705680 2-55834574863' 2026-02-17T04:46:15.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:46:15.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836496 2026-02-17T04:46:15.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:46:15.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:46:15.505 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-17T04:46:15.505 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:46:15.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836496 2026-02-17T04:46:15.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-17T04:46:15.507 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836496 2026-02-17T04:46:15.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:46:15.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836494 -lt 21474836496 2026-02-17T04:46:15.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:46:16.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:46:16.718 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:46:16.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836497 -lt 21474836496 2026-02-17T04:46:16.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:46:16.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705680 2026-02-17T04:46:16.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:46:16.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:46:16.937 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705680 2026-02-17T04:46:16.937 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:46:16.939 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705680 2026-02-17T04:46:16.939 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705680 2026-02-17T04:46:16.939 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T04:46:16.939 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:46:17.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T04:46:17.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:46:17.153 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574863 2026-02-17T04:46:17.153 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:46:17.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:46:17.155 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-17T04:46:17.155 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:46:17.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574863 2026-02-17T04:46:17.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-17T04:46:17.157 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574863 2026-02-17T04:46:17.157 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:46:17.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-17T04:46:17.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:46:17.378 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:46:17.378 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:46:17.378 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:46:17.379 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:46:17.380 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:46:17.380 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:46:17.380 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:46:17.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:46:17.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 2026-02-17T04:46:17.433 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:46:17.434 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 1000, 2026-02-17T04:46:17.434 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:46:17.434 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:46:17.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.441 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:46:17.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:46:17.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2026-02-17T04:46:17.461 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T04:46:17.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:46:17.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:46:17.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:46:17.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:46:17.463 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:46:17.463 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:46:17.463 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:46:17.464 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:46:17.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:46:17.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 2026-02-17T04:46:17.511 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:46:17.511 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 1000, 2026-02-17T04:46:17.511 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:46:17.511 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:46:17.518 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.518 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:46:17.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:46:17.529 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.529 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2026-02-17T04:46:17.539 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T04:46:17.539 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:46:17.541 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:46:17.541 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:46:17.542 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:46:17.542 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:46:17.542 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:46:17.542 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:46:17.542 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:46:17.542 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:46:17.543 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 0 2026-02-17T04:46:17.595 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:46:17.595 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 0, 2026-02-17T04:46:17.595 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [ 2026-02-17T04:46:17.595 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.595 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:12 2026", 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.171, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.210 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.196, 2026-02-17T04:46:17.596 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.242, 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.468 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.196, 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.242, 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.468 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.188 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:12 2026", 2026-02-17T04:46:17.597 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.157, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.208 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.188, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.254, 2026-02-17T04:46:17.598 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.501 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.188, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.254, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.501 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.201 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:15 2026", 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.599 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.147, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.159, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.196 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.219, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.248, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.479 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.600 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.219, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.248, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.479 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.157 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:15 2026", 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.601 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.602 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.146, 2026-02-17T04:46:17.602 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.147, 2026-02-17T04:46:17.602 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.167 2026-02-17T04:46:17.602 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.602 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.602 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.212, 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.263, 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.388 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.212, 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.263, 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.388 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.147 2026-02-17T04:46:17.603 INFO:tasks.workunit.client.0.trial105.stdout: } 2026-02-17T04:46:17.604 INFO:tasks.workunit.client.0.trial105.stdout: ] 2026-02-17T04:46:17.604 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:46:17.604 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.605 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:46:17.611 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2026-02-17T04:46:17.612 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.612 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2026-02-17T04:46:17.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:46:17.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:46:17.625 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:46:17.625 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:46:17.625 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:46:17.626 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:46:17.626 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:46:17.626 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:46:17.626 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:46:17.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:46:17.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 0 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 0, 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [ 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:12 2026", 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.673 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.171, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.210 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.159, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.142, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.134 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.674 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.196, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.242, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.468 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.243 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:12 2026", 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.675 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.157, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.208 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.135, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.126, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.114 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.188, 2026-02-17T04:46:17.676 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.254, 2026-02-17T04:46:17.677 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.501 2026-02-17T04:46:17.677 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.677 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.190 2026-02-17T04:46:17.677 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.677 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.677 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:15 2026", 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.147, 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.159, 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.196 2026-02-17T04:46:17.678 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.117, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.117, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.114 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.219, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.248, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.479 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.219 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.679 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:13 2026", 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.169, 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.184, 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.187 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.680 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.149, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.149, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.133 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.198, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.255, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.352 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.166 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:13 2026", 2026-02-17T04:46:17.681 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.155, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.175, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.186 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.140, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.126, 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.108 2026-02-17T04:46:17.682 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.190, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.279, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.633 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.159 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:13 2026", 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:46:17.683 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.162, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.158, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.180 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.153, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.108, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.108 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.185, 2026-02-17T04:46:17.684 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.226, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.292 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.154 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:15 2026", 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.146, 2026-02-17T04:46:17.685 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.147, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.167 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.112, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.112, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.110 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.212, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.263, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.388 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.686 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.212 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:13 2026", 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.687 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.688 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.142, 2026-02-17T04:46:17.688 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.145, 2026-02-17T04:46:17.688 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.165 2026-02-17T04:46:17.688 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.688 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.688 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.134, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.097, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.097 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.158, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.181, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.321 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.167 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.689 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:14 2026", 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.120, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.144, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.165 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.109, 2026-02-17T04:46:17.690 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.109, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.107 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.130, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.222, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.331 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.130 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:14 2026", 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.691 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.123, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.131, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.165 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.111, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.104, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.104 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.692 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.140, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.207, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.476 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.119 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:13 2026", 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.693 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.115, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.132, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.148 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.104, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.104, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.104 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.128, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.214, 2026-02-17T04:46:17.694 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.259 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.141 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:46:13 2026", 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.127, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.127, 2026-02-17T04:46:17.695 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.144 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.116, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.103, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.103 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.138, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.195, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.241 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.122 2026-02-17T04:46:17.696 INFO:tasks.workunit.client.0.trial105.stdout: } 2026-02-17T04:46:17.697 INFO:tasks.workunit.client.0.trial105.stdout: ] 2026-02-17T04:46:17.697 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:46:17.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:46:17.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2026-02-17T04:46:17.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:46:17.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2026-02-17T04:46:17.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:46:17.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2026-02-17T04:47:07.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2026-02-17T04:47:07.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:47:07.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:47:07.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:47:07.918 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:47:07.918 INFO:tasks.workunit.client.0.trial105.stderr:2' 2026-02-17T04:47:07.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:47:07.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:47:07.919 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:47:07.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836509 2026-02-17T04:47:07.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836509 2026-02-17T04:47:07.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836509' 2026-02-17T04:47:07.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:47:07.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:47:08.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705692 2026-02-17T04:47:08.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705692 2026-02-17T04:47:08.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836509 1-38654705692' 2026-02-17T04:47:08.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:47:08.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:47:08.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574875 2026-02-17T04:47:08.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574875 2026-02-17T04:47:08.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875' 2026-02-17T04:47:08.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:47:08.159 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836509 2026-02-17T04:47:08.159 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:47:08.160 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:47:08.161 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836509 2026-02-17T04:47:08.161 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:47:08.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836509 2026-02-17T04:47:08.163 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836509 2026-02-17T04:47:08.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-17T04:47:08.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:47:08.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836507 -lt 21474836509 2026-02-17T04:47:08.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:47:09.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:47:09.382 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:47:09.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836509 -lt 21474836509 2026-02-17T04:47:09.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:47:09.598 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705692 2026-02-17T04:47:09.598 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:47:09.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:47:09.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705692 2026-02-17T04:47:09.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:47:09.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705692 2026-02-17T04:47:09.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705692' 2026-02-17T04:47:09.602 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705692 2026-02-17T04:47:09.602 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:47:09.814 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705692 -lt 38654705692 2026-02-17T04:47:09.814 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:47:09.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574875 2026-02-17T04:47:09.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:47:09.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:47:09.817 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574875 2026-02-17T04:47:09.817 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:47:09.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574875 2026-02-17T04:47:09.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574875' 2026-02-17T04:47:09.819 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574875 2026-02-17T04:47:09.819 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:47:10.037 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574876 -lt 55834574875 2026-02-17T04:47:10.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:47:10.039 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:47:10.039 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:47:10.039 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:47:10.039 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:10.039 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:10.040 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:10.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:47:10.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:47:10.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 2026-02-17T04:47:10.094 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:47:10.094 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 1000, 2026-02-17T04:47:10.094 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:47:10.094 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:47:10.101 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.101 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:47:10.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:47:10.110 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.111 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2026-02-17T04:47:10.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T04:47:10.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:47:10.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:47:10.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:47:10.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:47:10.122 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:10.122 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:10.122 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:10.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:47:10.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:47:10.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 2026-02-17T04:47:10.169 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:47:10.169 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 1000, 2026-02-17T04:47:10.169 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:47:10.169 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:47:10.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:47:10.185 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:47:10.186 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.186 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2026-02-17T04:47:10.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T04:47:10.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:47:10.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:47:10.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:47:10.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:47:10.197 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:10.197 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:10.197 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:10.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:47:10.198 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:47:10.198 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 0 2026-02-17T04:47:10.250 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:47:10.250 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 0, 2026-02-17T04:47:10.250 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [ 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:07 2026", 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.165, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.179, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.179 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.177, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.205, 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.254 2026-02-17T04:47:10.251 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.177, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.205, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.254 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.270 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:07 2026", 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.252 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.159, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.166, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.177 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.178, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.196, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.251 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.178, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.196, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.251 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.220 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:05 2026", 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.135, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.137, 2026-02-17T04:47:10.253 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.150 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.148, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.248 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.148, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.248 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.196 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:05 2026", 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.141, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.143, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.146 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.158, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.163, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.263 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.254 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.158, 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.163, 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.263 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.235 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: } 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout: ] 2026-02-17T04:47:10.255 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:47:10.257 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.257 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:47:10.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2026-02-17T04:47:10.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2026-02-17T04:47:10.277 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:47:10.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:47:10.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:47:10.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:47:10.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 0 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 0, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [ 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:07 2026", 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.165, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.179, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.179 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.144, 2026-02-17T04:47:10.338 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.144, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.135 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.177, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.205, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.254 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.144 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:07 2026", 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.159, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.166, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.177 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.151, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.143, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.143 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.339 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.178, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.196, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.251 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.151 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:08 2026", 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.161, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.154, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.169 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.148, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.141, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.131 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.188, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.188, 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.298 2026-02-17T04:47:10.340 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.188 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:08 2026", 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.165, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.149, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.158 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.153, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.121, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.108 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.197, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.197, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.272 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.197 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.341 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:05 2026", 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.135, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.137, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.150 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.120, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.118, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.117 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.148, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.177, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.248 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.131 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:08 2026", 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.342 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.144, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.141, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.147 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.125, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.115, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.115 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.181, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.181, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.240 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.181 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:08 2026", 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 2, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.146, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.125, 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.146 2026-02-17T04:47:10.343 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.106, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.101, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.101 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.219, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.219, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.279 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.219 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:05 2026", 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 0, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.141, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.143, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.146 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.121, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.121, 2026-02-17T04:47:10.344 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.112 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.158, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.163, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.263 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.118 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:08 2026", 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 1, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.144, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.133, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.130 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.117, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.108, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.095 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.167, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.167, 2026-02-17T04:47:10.345 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.216 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.167 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:04 2026", 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.136, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.130, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.134 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.119, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.111, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.100 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.152, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.152, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.252 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.162 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.346 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:04 2026", 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 1, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "front", 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.120, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.114, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.129 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.103, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.100, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.094 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.144, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.146, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.222 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.170 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: { 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "last update": "Tue Feb 17 04:47:08 2026", 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "stale": false, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "from osd": 2, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "to osd": 0, 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "interface": "back", 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "average": { 2026-02-17T04:47:10.347 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.115, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.106, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.122 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "min": { 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.101, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.093, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.093 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "max": { 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "1min": 0.153, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "5min": 0.153, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "15min": 0.201 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: }, 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: "last": 0.153 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: } 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout: ] 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.348 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T04:47:10.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2026-02-17T04:47:10.356 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.356 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2026-02-17T04:47:10.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2026-02-17T04:47:10.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:10.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mgr.x.asok 2026-02-17T04:47:10.368 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:47:10.368 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-mgr.x.asok dump_osd_network 99 2026-02-17T04:47:10.414 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:47:10.414 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 99, 2026-02-17T04:47:10.414 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:47:10.414 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:47:10.421 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.421 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2026-02-17T04:47:10.431 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2026-02-17T04:47:10.432 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T04:47:10.432 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T04:47:10.432 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:47:10.432 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:47:10.433 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:10.433 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:10.433 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:10.433 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-osd.0.asok 2026-02-17T04:47:10.433 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T04:47:10.433 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.23392/ceph-osd.0.asok dump_osd_network 98 2026-02-17T04:47:10.488 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:47:10.488 INFO:tasks.workunit.client.0.trial105.stdout: "threshold": 98, 2026-02-17T04:47:10.488 INFO:tasks.workunit.client.0.trial105.stdout: "entries": [] 2026-02-17T04:47:10.488 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:47:10.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T04:47:10.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2026-02-17T04:47:10.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2026-02-17T04:47:10.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2026-02-17T04:47:10.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-17T04:47:10.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T04:47:10.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:47:10.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T04:47:10.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:47:10.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:47:10.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:47:10.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:47:10.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:47:10.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:47:10.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:47:10.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:47:10.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:47:10.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:47:10.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:47:10.822 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:47:10.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:47:10.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:47:10.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:47:10.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:47:10.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:47:10.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:47:10.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T04:47:11.396 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:47:11.396 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.396 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.396 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.23392 2026-02-17T04:47:11.398 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:47:11.398 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:47:11.398 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-17T04:47:11.398 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:47:11.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:47:11.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:47:11.401 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:47:11.401 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:47:11.402 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:47:11.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:47:11.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:47:11.403 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:47:11.404 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:47:11.404 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:47:11.404 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:47:11.404 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:47:11.405 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:47:11.406 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:47:11.406 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T04:47:11.408 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:47:11.408 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.408 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.408 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.23392 2026-02-17T04:47:11.409 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:47:11.409 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:47:11.409 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-17T04:47:11.410 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:47:11.411 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.411 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.411 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.23392 2026-02-17T04:47:11.412 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test2 td/network-ping 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:111: TEST_network_ping_test2: local dir=td/network-ping 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:113: TEST_network_ping_test2: export CEPH_ARGS 2026-02-17T04:47:11.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: export 'EXTRA_OPTS= --mon_warn_on_slow_ping_time=0.001' 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: EXTRA_OPTS=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:115: TEST_network_ping_test2: run_mon td/network-ping a 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-17T04:47:11.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-17T04:47:11.437 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:47:11.437 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:47:11.437 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:47:11.437 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:47:11.437 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.437 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.438 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:11.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --mon_warn_on_slow_ping_time=0.001 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:47:11.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:47:11.468 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:47:11.468 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:47:11.468 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:47:11.469 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.470 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.471 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mon.a.asok 2026-02-17T04:47:11.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:47:11.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.23392/ceph-mon.a.asok config get fsid 2026-02-17T04:47:11.525 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:47:11.526 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:47:11.527 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:47:11.527 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.527 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.527 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.23392/ceph-mon.a.asok 2026-02-17T04:47:11.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:47:11.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.23392/ceph-mon.a.asok config get mon_host 2026-02-17T04:47:11.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:116: TEST_network_ping_test2: run_mgr td/network-ping x 2026-02-17T04:47:11.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-17T04:47:11.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:47:11.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:47:11.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:47:11.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-17T04:47:11.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:47:11.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:47:11.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:47:11.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:47:11.701 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:47:11.701 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.701 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.702 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:11.702 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:47:11.703 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --mon_warn_on_slow_ping_time=0.001 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:117: TEST_network_ping_test2: run_osd td/network-ping 0 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T04:47:11.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:11.724 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:47:11.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:47:11.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-17T04:47:11.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:47:11.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=45136630-cd90-4e20-8c24-551bc3719a13 2026-02-17T04:47:11.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 45136630-cd90-4e20-8c24-551bc3719a13' 2026-02-17T04:47:11.727 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 45136630-cd90-4e20-8c24-551bc3719a13 2026-02-17T04:47:11.727 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:47:11.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDP8pNpspGEKxAAQVtb1K3tnGswWGjwRVl9+Q== 2026-02-17T04:47:11.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDP8pNpspGEKxAAQVtb1K3tnGswWGjwRVl9+Q=="}' 2026-02-17T04:47:11.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 45136630-cd90-4e20-8c24-551bc3719a13 -i td/network-ping/0/new.json 2026-02-17T04:47:11.849 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:47:11.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-17T04:47:11.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDP8pNpspGEKxAAQVtb1K3tnGswWGjwRVl9+Q== --osd-uuid 45136630-cd90-4e20-8c24-551bc3719a13 2026-02-17T04:47:11.874 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:11.867+0000 7f9fd4171a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:11.875 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:11.867+0000 7f9fd4171a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:11.876 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:11.867+0000 7f9fd4171a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:11.877 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:11.871+0000 7f9fd4171a40 -1 bdev(0x5629cb05c400 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:47:11.877 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:11.871+0000 7f9fd4171a40 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-17T04:47:12.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-17T04:47:12.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:47:12.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:47:12.698 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:47:12.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:47:12.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:47:12.818 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:47:12.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:47:12.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:47:12.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:47:12.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:47:12.843 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:12.835+0000 7f0278043a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:12.845 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:12.839+0000 7f0278043a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:12.847 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:12.839+0000 7f0278043a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:12.927 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:47:12.927 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:47:12.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:47:12.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:47:12.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:47:12.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:12.928 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:47:12.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:47:12.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:12.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:47:13.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:47:13.430 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:13.423+0000 7f0278043a40 -1 Falling back to public interface 2026-02-17T04:47:14.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:47:14.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:14.040 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:47:14.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:47:14.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:14.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:47:14.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:47:14.373 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:14.367+0000 7f0278043a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:47:15.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:47:15.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:15.255 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:47:15.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:47:15.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:15.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:47:15.476 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4150932014,v1:127.0.0.1:6801/4150932014] [v2:127.0.0.1:6802/4150932014,v1:127.0.0.1:6803/4150932014] exists,up 45136630-cd90-4e20-8c24-551bc3719a13 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:118: TEST_network_ping_test2: run_osd td/network-ping 1 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:47:15.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:15.478 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-17T04:47:15.479 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:47:15.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f5b4942d-f54b-404f-9af6-802ed9a43ee8 2026-02-17T04:47:15.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f5b4942d-f54b-404f-9af6-802ed9a43ee8' 2026-02-17T04:47:15.481 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 f5b4942d-f54b-404f-9af6-802ed9a43ee8 2026-02-17T04:47:15.481 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:47:15.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDT8pNpBzMnHRAAE9rR9iy5CpIbUhSLHqFQpA== 2026-02-17T04:47:15.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDT8pNpBzMnHRAAE9rR9iy5CpIbUhSLHqFQpA=="}' 2026-02-17T04:47:15.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f5b4942d-f54b-404f-9af6-802ed9a43ee8 -i td/network-ping/1/new.json 2026-02-17T04:47:15.704 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:47:15.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-17T04:47:15.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDT8pNpBzMnHRAAE9rR9iy5CpIbUhSLHqFQpA== --osd-uuid f5b4942d-f54b-404f-9af6-802ed9a43ee8 2026-02-17T04:47:15.731 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:15.723+0000 7f4bda888a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:15.732 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:15.727+0000 7f4bda888a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:15.733 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:15.727+0000 7f4bda888a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:15.734 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:15.727+0000 7f4bda888a40 -1 bdev(0x5614f7b6a400 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:47:15.734 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:15.727+0000 7f4bda888a40 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-17T04:47:16.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-17T04:47:16.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:47:16.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:47:16.376 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:47:16.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:47:16.663 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:47:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:47:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:47:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:47:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:47:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:47:16.679 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:16.671+0000 7f0d15719a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:16.680 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:16.675+0000 7f0d15719a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:16.681 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:16.675+0000 7f0d15719a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:16.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:47:16.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:47:16.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:47:16.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:47:16.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:47:16.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:16.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:47:16.883 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:47:16.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:16.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:47:17.074 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:17.067+0000 7f0d15719a40 -1 Falling back to public interface 2026-02-17T04:47:17.098 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:47:17.664 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:17.655+0000 7f0d15719a40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:47:18.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:47:18.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:18.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:47:18.099 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:47:18.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:18.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:47:18.310 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1365375731,v1:127.0.0.1:6811/1365375731] [v2:127.0.0.1:6812/1365375731,v1:127.0.0.1:6813/1365375731] exists,up f5b4942d-f54b-404f-9af6-802ed9a43ee8 2026-02-17T04:47:18.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:47:18.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:47:18.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:47:18.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:119: TEST_network_ping_test2: run_osd td/network-ping 2 2026-02-17T04:47:18.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T04:47:18.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:47:18.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T04:47:18.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:47:18.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-17T04:47:18.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T04:47:18.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:47:18.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:47:18.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:47:18.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-17T04:47:18.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-17T04:47:18.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:47:18.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T04:47:18.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T04:47:18.321 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:47:18.321 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:47:18.321 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:47:18.321 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:47:18.321 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:47:18.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T04:47:18.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T04:47:18.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:47:18.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:47:18.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:47:18.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:47:18.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:47:18.324 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:47:18.324 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-17T04:47:18.324 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:47:18.324 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=165666ec-6fa6-4337-a59f-e5c73dbe6446 2026-02-17T04:47:18.325 INFO:tasks.workunit.client.0.trial105.stdout:add osd2 165666ec-6fa6-4337-a59f-e5c73dbe6446 2026-02-17T04:47:18.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 165666ec-6fa6-4337-a59f-e5c73dbe6446' 2026-02-17T04:47:18.325 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:47:18.328 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDW8pNp1w5BExAA+Togl8lX3fatpshQuf6VIQ== 2026-02-17T04:47:18.328 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDW8pNp1w5BExAA+Togl8lX3fatpshQuf6VIQ=="}' 2026-02-17T04:47:18.328 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 165666ec-6fa6-4337-a59f-e5c73dbe6446 -i td/network-ping/2/new.json 2026-02-17T04:47:18.534 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:47:18.543 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-17T04:47:18.544 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDW8pNp1w5BExAA+Togl8lX3fatpshQuf6VIQ== --osd-uuid 165666ec-6fa6-4337-a59f-e5c73dbe6446 2026-02-17T04:47:18.562 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:18.556+0000 7ff16ffc2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:18.563 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:18.556+0000 7ff16ffc2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:18.564 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:18.560+0000 7ff16ffc2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:18.565 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:18.560+0000 7ff16ffc2a40 -1 bdev(0x55b30d696400 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-17T04:47:18.565 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:18.560+0000 7ff16ffc2a40 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-17T04:47:19.214 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-17T04:47:19.215 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:47:19.215 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T04:47:19.216 INFO:tasks.workunit.client.0.trial105.stdout:adding osd2 key to auth repository 2026-02-17T04:47:19.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:47:19.517 INFO:tasks.workunit.client.0.trial105.stdout:start osd.2 2026-02-17T04:47:19.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T04:47:19.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=681f3b51-d246-40f1-a434-16329859919d --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.23392/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:47:19.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:47:19.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:47:19.519 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:47:19.534 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:19.528+0000 7f15498c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:19.536 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:19.528+0000 7f15498c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:19.537 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:19.532+0000 7f15498c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:19.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:47:19.739 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:47:19.739 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:19.739 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:47:19.909 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:19.904+0000 7f15498c0a40 -1 Falling back to public interface 2026-02-17T04:47:19.955 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:47:20.585 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:47:20.580+0000 7f15498c0a40 -1 osd.2 0 log_to_monitors true 2026-02-17T04:47:20.956 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:47:20.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:20.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:47:20.957 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:47:20.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:20.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:47:21.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:47:22.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:47:22.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:47:22.168 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:47:22.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:47:22.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:47:22.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:47:22.376 INFO:tasks.workunit.client.0.trial105.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2727458709,v1:127.0.0.1:6819/2727458709] [v2:127.0.0.1:6820/2727458709,v1:127.0.0.1:6821/2727458709] exists,up 165666ec-6fa6-4337-a59f-e5c73dbe6446 2026-02-17T04:47:22.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:47:22.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:47:22.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:47:22.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:121: TEST_network_ping_test2: sleep 5 2026-02-17T04:47:27.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:122: TEST_network_ping_test2: ceph osd crush add-bucket dc1 datacenter 2026-02-17T04:47:27.629 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'dc1' already exists 2026-02-17T04:47:27.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:123: TEST_network_ping_test2: ceph osd crush add-bucket dc2 datacenter 2026-02-17T04:47:27.904 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'dc2' already exists 2026-02-17T04:47:27.913 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:124: TEST_network_ping_test2: ceph osd crush add-bucket dc3 datacenter 2026-02-17T04:47:28.210 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'dc3' already exists 2026-02-17T04:47:28.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:125: TEST_network_ping_test2: ceph osd crush add-bucket rack1 rack 2026-02-17T04:47:28.484 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'rack1' already exists 2026-02-17T04:47:28.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:126: TEST_network_ping_test2: ceph osd crush add-bucket rack2 rack 2026-02-17T04:47:28.765 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'rack2' already exists 2026-02-17T04:47:28.775 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:127: TEST_network_ping_test2: ceph osd crush add-bucket rack3 rack 2026-02-17T04:47:29.032 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'rack3' already exists 2026-02-17T04:47:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:128: TEST_network_ping_test2: ceph osd crush add-bucket host1 host 2026-02-17T04:47:29.314 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'host1' already exists 2026-02-17T04:47:29.324 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:129: TEST_network_ping_test2: ceph osd crush add-bucket host2 host 2026-02-17T04:47:29.581 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'host2' already exists 2026-02-17T04:47:29.590 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:130: TEST_network_ping_test2: ceph osd crush add-bucket host3 host 2026-02-17T04:47:29.862 INFO:tasks.workunit.client.0.trial105.stderr:bucket 'host3' already exists 2026-02-17T04:47:29.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:131: TEST_network_ping_test2: ceph osd crush move dc1 root=default 2026-02-17T04:47:30.167 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -5 name 'dc1' to location {root=default} in crush map 2026-02-17T04:47:30.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:132: TEST_network_ping_test2: ceph osd crush move dc2 root=default 2026-02-17T04:47:30.446 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -6 name 'dc2' to location {root=default} in crush map 2026-02-17T04:47:30.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:133: TEST_network_ping_test2: ceph osd crush move dc3 root=default 2026-02-17T04:47:30.725 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -7 name 'dc3' to location {root=default} in crush map 2026-02-17T04:47:30.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:134: TEST_network_ping_test2: ceph osd crush move rack1 datacenter=dc1 2026-02-17T04:47:30.998 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -8 name 'rack1' to location {datacenter=dc1} in crush map 2026-02-17T04:47:31.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:135: TEST_network_ping_test2: ceph osd crush move rack2 datacenter=dc2 2026-02-17T04:47:31.274 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -9 name 'rack2' to location {datacenter=dc2} in crush map 2026-02-17T04:47:31.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:136: TEST_network_ping_test2: ceph osd crush move rack3 datacenter=dc3 2026-02-17T04:47:31.560 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -10 name 'rack3' to location {datacenter=dc3} in crush map 2026-02-17T04:47:31.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:137: TEST_network_ping_test2: ceph osd crush move host1 rack=rack1 2026-02-17T04:47:31.828 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -11 name 'host1' to location {rack=rack1} in crush map 2026-02-17T04:47:31.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:138: TEST_network_ping_test2: ceph osd crush move host2 rack=rack2 2026-02-17T04:47:32.132 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -12 name 'host2' to location {rack=rack2} in crush map 2026-02-17T04:47:32.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:139: TEST_network_ping_test2: ceph osd crush move host3 rack=rack3 2026-02-17T04:47:32.402 INFO:tasks.workunit.client.0.trial105.stderr:no need to move item id -13 name 'host3' to location {rack=rack3} in crush map 2026-02-17T04:47:32.412 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:140: TEST_network_ping_test2: ceph osd crush set osd.0 1.0 host=host1 2026-02-17T04:47:32.663 INFO:tasks.workunit.client.0.trial105.stderr:set item id 0 name 'osd.0' weight 1 at location {host=host1}: no change 2026-02-17T04:47:32.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:141: TEST_network_ping_test2: ceph osd crush set osd.1 1.0 host=host2 2026-02-17T04:47:32.967 INFO:tasks.workunit.client.0.trial105.stderr:set item id 1 name 'osd.1' weight 1 at location {host=host2}: no change 2026-02-17T04:47:32.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:142: TEST_network_ping_test2: ceph osd crush set osd.2 1.0 host=host3 2026-02-17T04:47:33.243 INFO:tasks.workunit.client.0.trial105.stderr:set item id 2 name 'osd.2' weight 1 at location {host=host3}: no change 2026-02-17T04:47:33.252 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:143: TEST_network_ping_test2: ceph osd crush rule create-simple myrule default host firstn 2026-02-17T04:47:33.520 INFO:tasks.workunit.client.0.trial105.stderr:rule myrule already exists 2026-02-17T04:47:33.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:145: TEST_network_ping_test2: create_pool foo 16 16 replicated myrule 2026-02-17T04:47:33.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 16 replicated myrule 2026-02-17T04:47:33.803 INFO:tasks.workunit.client.0.trial105.stderr:pool 'foo' already exists 2026-02-17T04:47:33.813 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:47:34.814 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:148: TEST_network_ping_test2: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-17T04:47:34.835 INFO:tasks.workunit.client.0.trial105.stdout:hints = 1 2026-02-17T04:47:34.836 INFO:tasks.workunit.client.0.trial105.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-17T04:47:34.836 INFO:tasks.workunit.client.0.trial105.stdout:Object prefix: benchmark_data_trial105_34803 2026-02-17T04:47:34.836 INFO:tasks.workunit.client.0.trial105.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T04:47:34.836 INFO:tasks.workunit.client.0.trial105.stdout: 0 0 0 0 0 0 - 0 2026-02-17T04:47:35.836 INFO:tasks.workunit.client.0.trial105.stdout: 1 16 10967 10951 42.7741 42.7773 0.00106783 0.00145872 2026-02-17T04:47:36.836 INFO:tasks.workunit.client.0.trial105.stdout: 2 15 22013 21998 42.9616 43.1523 0.0010201 0.00145288 2026-02-17T04:47:37.836 INFO:tasks.workunit.client.0.trial105.stdout: 3 16 33051 33035 43.0111 43.1133 0.000873899 0.00145149 2026-02-17T04:47:38.835 INFO:tasks.workunit.client.0.trial105.stdout: 4 16 44622 44606 43.5572 45.1992 0.00139417 0.00143326 2026-02-17T04:47:39.836 INFO:tasks.workunit.client.0.trial105.stdout: 5 16 55908 55892 43.6623 44.0859 0.00081295 0.00142975 2026-02-17T04:47:40.836 INFO:tasks.workunit.client.0.trial105.stdout: 6 16 67042 67026 43.6334 43.4922 0.0012196 0.0014308 2026-02-17T04:47:41.836 INFO:tasks.workunit.client.0.trial105.stdout: 7 16 78214 78198 43.6339 43.6406 0.00138841 0.0014308 2026-02-17T04:47:42.836 INFO:tasks.workunit.client.0.trial105.stdout: 8 16 88522 88506 43.2124 40.2656 0.00115004 0.00144484 2026-02-17T04:47:43.836 INFO:tasks.workunit.client.0.trial105.stdout: 9 16 98145 98129 42.5874 37.5898 0.00179809 0.00146604 2026-02-17T04:47:44.836 INFO:tasks.workunit.client.0.trial105.stdout: 10 3 107829 107826 42.1162 37.8789 0.00111303 0.00148248 2026-02-17T04:47:45.836 INFO:tasks.workunit.client.0.trial105.stdout:Total time run: 10.0009 2026-02-17T04:47:45.836 INFO:tasks.workunit.client.0.trial105.stdout:Total writes made: 107829 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Write size: 4096 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Object size: 4096 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Bandwidth (MB/sec): 42.1168 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Stddev Bandwidth: 2.62412 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Max bandwidth (MB/sec): 45.1992 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Min bandwidth (MB/sec): 37.5898 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Average IOPS: 10781 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Stddev IOPS: 671.775 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Max IOPS: 11571 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Min IOPS: 9623 2026-02-17T04:47:45.837 INFO:tasks.workunit.client.0.trial105.stdout:Average Latency(s): 0.00148247 2026-02-17T04:47:45.838 INFO:tasks.workunit.client.0.trial105.stdout:Stddev Latency(s): 0.000755525 2026-02-17T04:47:45.838 INFO:tasks.workunit.client.0.trial105.stdout:Max latency(s): 0.0162536 2026-02-17T04:47:45.838 INFO:tasks.workunit.client.0.trial105.stdout:Min latency(s): 0.000451294 2026-02-17T04:47:45.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:151: TEST_network_ping_test2: sleep 10 2026-02-17T04:47:55.844 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:152: TEST_network_ping_test2: flush_pg_stats 2026-02-17T04:47:55.844 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:47:55.845 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:47:56.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:47:56.052 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:47:56.052 INFO:tasks.workunit.client.0.trial105.stderr:2' 2026-02-17T04:47:56.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:47:56.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:47:56.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:47:56.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-17T04:47:56.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-17T04:47:56.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T04:47:56.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:47:56.132 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:47:56.210 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-17T04:47:56.210 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-17T04:47:56.210 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-38654705674' 2026-02-17T04:47:56.210 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:47:56.210 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:47:56.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-17T04:47:56.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-17T04:47:56.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857' 2026-02-17T04:47:56.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:47:56.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836491 2026-02-17T04:47:56.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:47:56.292 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:47:56.293 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-17T04:47:56.293 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:47:56.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836491 2026-02-17T04:47:56.295 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836491 2026-02-17T04:47:56.295 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T04:47:56.295 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:47:56.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836491 2026-02-17T04:47:56.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:47:57.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:47:57.502 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:47:57.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836491 2026-02-17T04:47:57.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:47:58.719 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T04:47:58.719 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:47:58.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T04:47:58.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:47:58.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705674 2026-02-17T04:47:58.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:47:58.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:47:58.936 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-17T04:47:58.936 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:47:58.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705674 2026-02-17T04:47:58.938 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705674 2026-02-17T04:47:58.938 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T04:47:58.938 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:47:59.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T04:47:59.160 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:47:59.160 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574857 2026-02-17T04:47:59.160 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:47:59.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:47:59.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574857 2026-02-17T04:47:59.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:47:59.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-17T04:47:59.164 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574857 2026-02-17T04:47:59.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T04:47:59.164 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:47:59.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-17T04:47:59.387 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: ceph health 2026-02-17T04:47:59.387 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: tee td/network-ping/health 2026-02-17T04:47:59.627 INFO:tasks.workunit.client.0.trial105.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.302ms); Slow OSD heartbeats on front (longest 0.321ms) 2026-02-17T04:47:59.636 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:155: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats' td/network-ping/health 2026-02-17T04:47:59.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: ceph health detail 2026-02-17T04:47:59.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: tee td/network-ping/health 2026-02-17T04:47:59.925 INFO:tasks.workunit.client.0.trial105.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.302ms); Slow OSD heartbeats on front (longest 0.321ms) 2026-02-17T04:47:59.925 INFO:tasks.workunit.client.0.trial105.stdout:[WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 0.302ms) 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.302 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.261 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.203 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.181 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.178 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.170 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout:[WRN] OSD_SLOW_PING_TIME_FRONT: Slow OSD heartbeats on front (longest 0.321ms) 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.321 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.283 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.232 msec 2026-02-17T04:47:59.926 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.218 msec possibly improving 2026-02-17T04:47:59.927 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.202 msec 2026-02-17T04:47:59.927 INFO:tasks.workunit.client.0.trial105.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.184 msec possibly improving 2026-02-17T04:47:59.936 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:158: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_BACK td/network-ping/health 2026-02-17T04:47:59.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:159: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_FRONT td/network-ping/health 2026-02-17T04:47:59.938 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:160: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats on front from osd[.][0-2] [[]dc[1-3],rack[1-3][]] to osd[.][0-2] [[]dc[1-3],rack[1-3][]]' td/network-ping/health 2026-02-17T04:47:59.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:162: TEST_network_ping_test2: rm -f td/network-ping/health 2026-02-17T04:47:59.941 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-17T04:47:59.941 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T04:47:59.942 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:47:59.942 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T04:47:59.943 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:47:59.943 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:47:59.943 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:47:59.943 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:47:59.943 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:00.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:00.253 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:00.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:00.255 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:00.257 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:00.257 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:00.257 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:00.258 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:00.258 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:00.259 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:00.259 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:00.260 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:00.261 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:00.262 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T04:48:00.851 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:00.851 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:00.852 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:48:00.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.23392 2026-02-17T04:48:00.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:00.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/network-ping 0 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:00.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:00.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:00.855 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:00.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:00.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:00.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:00.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:00.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:00.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:00.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:00.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:00.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:00.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:00.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T04:48:00.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T04:48:00.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:00.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:00.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.23392 2026-02-17T04:48:00.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.23392 2026-02-17T04:48:00.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:00.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:00.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T04:48:00.867 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T04:48:00.868 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T04:48:00.918 INFO:tasks.workunit:Running workunit misc/ok-to-stop.sh... 2026-02-17T04:48:00.918 DEBUG:teuthology.orchestra.run.trial105:workunit test misc/ok-to-stop.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh 2026-02-17T04:48:00.972 INFO:tasks.workunit.client.0.trial105.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T04:48:00.975 INFO:tasks.workunit.client.0.trial105.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T04:48:00.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/ok-to-stop 2026-02-17T04:48:00.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:6: run: local dir=td/ok-to-stop 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:7: run: shift 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: export CEPH_MON_A=127.0.0.1:7150 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: CEPH_MON_A=127.0.0.1:7150 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: export CEPH_MON_B=127.0.0.1:7151 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: CEPH_MON_B=127.0.0.1:7151 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: export CEPH_MON_C=127.0.0.1:7152 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: CEPH_MON_C=127.0.0.1:7152 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: export CEPH_MON_D=127.0.0.1:7153 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: CEPH_MON_D=127.0.0.1:7153 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: export CEPH_MON_E=127.0.0.1:7154 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: CEPH_MON_E=127.0.0.1:7154 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:14: run: export CEPH_ARGS 2026-02-17T04:48:00.977 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: uuidgen 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: CEPH_ARGS+='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none ' 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: export 'ORIG_CEPH_ARGS=--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none ' 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: ORIG_CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none ' 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: set 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: local 'funcs=TEST_0_mds 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:TEST_0_osd 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:TEST_1_mon_checks 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:TEST_2_mons_checks 2026-02-17T04:48:00.980 INFO:tasks.workunit.client.0.trial105.stderr:TEST_3_mons_checks 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:TEST_4_mons_checks 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:TEST_5_mons_checks' 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:00.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:00.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:00.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:00.982 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:00.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:00.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:00.984 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:00.984 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:00.984 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:00.984 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:00.985 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:00.985 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:00.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:00.986 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:00.987 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:00.987 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:00.989 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:00.989 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:00.989 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:00.989 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:00.989 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:00.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:00.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:48:00.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:48:00.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:00.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:00.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:48:00.992 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_mds td/ok-to-stop 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:208: TEST_0_mds: local dir=td/ok-to-stop 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:210: TEST_0_mds: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:212: TEST_0_mds: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:00.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:00.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:48:00.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:00.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:48:00.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:48:01.020 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:01.020 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:01.020 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:01.020 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:01.020 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:01.020 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:01.021 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:01.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:48:01.049 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:01.052 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:01.053 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:01.053 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:01.053 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:01.053 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:01.053 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:48:01.107 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:01.108 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:01.109 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:01.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:01.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:48:01.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:213: TEST_0_mds: run_mgr td/ok-to-stop x 2026-02-17T04:48:01.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-17T04:48:01.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:48:01.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:48:01.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:48:01.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-17T04:48:01.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:48:01.277 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:48:01.277 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:01.277 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:01.277 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:01.278 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:01.278 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:01.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:01.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:48:01.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:48:01.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:214: TEST_0_mds: run_osd td/ok-to-stop 0 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:48:01.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:01.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:48:01.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:48:01.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-17T04:48:01.302 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:48:01.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0dc8293d-b479-4184-91ad-90b62b3a2585 2026-02-17T04:48:01.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0dc8293d-b479-4184-91ad-90b62b3a2585' 2026-02-17T04:48:01.303 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 0dc8293d-b479-4184-91ad-90b62b3a2585 2026-02-17T04:48:01.303 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:48:01.311 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAB85NpSu9iEhAANsTgoa553+uguIUOchcTvg== 2026-02-17T04:48:01.312 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAB85NpSu9iEhAANsTgoa553+uguIUOchcTvg=="}' 2026-02-17T04:48:01.312 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0dc8293d-b479-4184-91ad-90b62b3a2585 -i td/ok-to-stop/0/new.json 2026-02-17T04:48:01.420 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:01.428 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-17T04:48:01.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAB85NpSu9iEhAANsTgoa553+uguIUOchcTvg== --osd-uuid 0dc8293d-b479-4184-91ad-90b62b3a2585 2026-02-17T04:48:01.444 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:01.436+0000 7f8d0861aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:01.446 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:01.436+0000 7f8d0861aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:01.447 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:01.440+0000 7f8d0861aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:01.447 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:01.440+0000 7f8d0861aa40 -1 bdev(0x5628036e2400 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:48:01.448 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:01.440+0000 7f8d0861aa40 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-17T04:48:02.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-17T04:48:02.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:48:02.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:48:02.158 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:48:02.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:48:02.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:48:02.268 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:48:02.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:48:02.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:48:02.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:48:02.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:48:02.282 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:02.276+0000 7fa73b3aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:02.284 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:02.276+0000 7fa73b3aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:02.285 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:02.276+0000 7fa73b3aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:02.380 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:02.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:48:02.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:02.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:48:02.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:02.692 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:02.684+0000 7fa73b3aca40 -1 Falling back to public interface 2026-02-17T04:48:03.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:03.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:03.492 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:48:03.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:48:03.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:03.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:48:03.510 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:03.505+0000 7fa73b3aca40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:48:03.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:04.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:04.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:04.606 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:48:04.606 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:48:04.606 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:04.606 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/167366287,v1:127.0.0.1:6801/167366287] [v2:127.0.0.1:6802/167366287,v1:127.0.0.1:6803/167366287] exists,up 0dc8293d-b479-4184-91ad-90b62b3a2585 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:215: TEST_0_mds: run_mds td/ok-to-stop a 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=a 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-17T04:48:04.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/a 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:04.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id a --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:04.838 INFO:tasks.workunit.client.0.trial105.stdout:starting mds.a at 2026-02-17T04:48:04.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:217: TEST_0_mds: ceph osd pool create meta 1 2026-02-17T04:48:05.116 INFO:tasks.workunit.client.0.trial105.stderr:pool 'meta' already exists 2026-02-17T04:48:05.125 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:218: TEST_0_mds: ceph osd pool create data 1 2026-02-17T04:48:05.420 INFO:tasks.workunit.client.0.trial105.stderr:pool 'data' already exists 2026-02-17T04:48:05.430 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:219: TEST_0_mds: ceph fs new myfs meta data 2026-02-17T04:48:05.775 INFO:tasks.workunit.client.0.trial105.stderr:filesystem 'myfs' already exists 2026-02-17T04:48:05.786 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:220: TEST_0_mds: sleep 5 2026-02-17T04:48:10.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:222: TEST_0_mds: ceph mds ok-to-stop a 2026-02-17T04:48:10.973 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 2026-02-17T04:48:10.978 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:223: TEST_0_mds: ceph mds ok-to-stop a dne 2026-02-17T04:48:11.167 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 2026-02-17T04:48:11.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:224: TEST_0_mds: ceph mds ok-to-stop dne 2026-02-17T04:48:11.450 INFO:tasks.workunit.client.0.trial105.stderr:should be safe to stop [dne] 2026-02-17T04:48:11.461 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:226: TEST_0_mds: run_mds td/ok-to-stop b 2026-02-17T04:48:11.461 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-17T04:48:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-17T04:48:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=b 2026-02-17T04:48:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-17T04:48:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/b 2026-02-17T04:48:11.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-17T04:48:11.464 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:11.464 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:11.464 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:11.464 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:11.464 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:11.464 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:11.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id b --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:11.487 INFO:tasks.workunit.client.0.trial105.stdout:starting mds.b at 2026-02-17T04:48:11.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:227: TEST_0_mds: sleep 5 2026-02-17T04:48:16.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:229: TEST_0_mds: ceph mds ok-to-stop a 2026-02-17T04:48:16.771 INFO:tasks.workunit.client.0.trial105.stderr:should be safe to stop [a] 2026-02-17T04:48:16.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:230: TEST_0_mds: ceph mds ok-to-stop b 2026-02-17T04:48:17.063 INFO:tasks.workunit.client.0.trial105.stderr:should be safe to stop [b] 2026-02-17T04:48:17.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:231: TEST_0_mds: ceph mds ok-to-stop a b 2026-02-17T04:48:17.265 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 4137 2026-02-17T04:48:17.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:232: TEST_0_mds: ceph mds ok-to-stop a dne1 dne2 2026-02-17T04:48:17.552 INFO:tasks.workunit.client.0.trial105.stderr:should be safe to stop [a,dne1,dne2] 2026-02-17T04:48:17.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:233: TEST_0_mds: ceph mds ok-to-stop b dne 2026-02-17T04:48:17.851 INFO:tasks.workunit.client.0.trial105.stderr:should be safe to stop [b,dne] 2026-02-17T04:48:17.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:234: TEST_0_mds: ceph mds ok-to-stop a b dne 2026-02-17T04:48:18.060 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 4137 2026-02-17T04:48:18.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:235: TEST_0_mds: ceph mds ok-to-stop dne1 dne2 2026-02-17T04:48:18.334 INFO:tasks.workunit.client.0.trial105.stderr:should be safe to stop [dne1,dne2] 2026-02-17T04:48:18.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:237: TEST_0_mds: kill_daemons td/ok-to-stop KILL mds.a 2026-02-17T04:48:18.346 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:18.346 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:18.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:18.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:18.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:18.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:18.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:18.454 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:18.454 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:18.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:18.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:18.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:18.563 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:18.563 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:18.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:18.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:18.569 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:18.570 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:18.570 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:18.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:18.572 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:18.572 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:18.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:18.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:18.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:18.573 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:18.574 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:18.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:18.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:18.581 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:18.581 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.581 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:18.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:18.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:18.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:18.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:18.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:18.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:18.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:18.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:18.587 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:18.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:18.587 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:18.588 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:18.589 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:18.589 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:18.590 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:18.590 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:18.590 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:18.590 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:18.591 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:18.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:18.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:18.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:18.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:18.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:18.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:18.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:48:18.596 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:48:18.596 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.597 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:48:18.598 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:48:18.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_osd td/ok-to-stop 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:241: TEST_0_osd: local dir=td/ok-to-stop 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:243: TEST_0_osd: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:245: TEST_0_osd: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:48:18.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:18.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:48:18.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:48:18.620 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:18.620 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:18.621 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:18.621 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:18.621 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.621 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.621 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:18.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:48:18.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:18.650 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:48:18.650 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:18.650 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:18.650 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:18.651 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:18.651 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:18.652 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:18.653 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:48:18.708 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:18.709 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:18.710 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:18.710 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.710 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.711 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:18.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:18.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:246: TEST_0_osd: run_mgr td/ok-to-stop x 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-17T04:48:18.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.880 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:18.881 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:48:18.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:48:18.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:247: TEST_0_osd: run_osd td/ok-to-stop 0 2026-02-17T04:48:18.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T04:48:18.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:18.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:48:18.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-17T04:48:18.902 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:48:18.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=982d1365-09e1-467c-bc52-574fe278ae5c 2026-02-17T04:48:18.903 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 982d1365-09e1-467c-bc52-574fe278ae5c 2026-02-17T04:48:18.903 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 982d1365-09e1-467c-bc52-574fe278ae5c' 2026-02-17T04:48:18.903 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:48:18.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAS85NpOuhiNhAA/FdZl/E04HAveKRoF8FumQ== 2026-02-17T04:48:18.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAS85NpOuhiNhAA/FdZl/E04HAveKRoF8FumQ=="}' 2026-02-17T04:48:18.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 982d1365-09e1-467c-bc52-574fe278ae5c -i td/ok-to-stop/0/new.json 2026-02-17T04:48:19.026 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:19.034 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-17T04:48:19.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAS85NpOuhiNhAA/FdZl/E04HAveKRoF8FumQ== --osd-uuid 982d1365-09e1-467c-bc52-574fe278ae5c 2026-02-17T04:48:19.049 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.045+0000 7f21a2ae3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:19.051 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.045+0000 7f21a2ae3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:19.052 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.045+0000 7f21a2ae3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:19.053 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.045+0000 7f21a2ae3a40 -1 bdev(0x55e57b088400 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:48:19.053 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.045+0000 7f21a2ae3a40 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-17T04:48:19.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-17T04:48:19.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:48:19.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:48:19.853 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:48:19.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:48:19.970 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:48:19.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:48:19.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:48:19.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:48:19.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:48:19.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:48:19.993 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.989+0000 7f4472953a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:19.995 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.989+0000 7f4472953a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:19.997 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:19.989+0000 7f4472953a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:20.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:48:20.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:48:20.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:48:20.079 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:48:20.079 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:48:20.079 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:20.079 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:20.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:48:20.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:20.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:48:20.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:20.542 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:20.537+0000 7f4472953a40 -1 Falling back to public interface 2026-02-17T04:48:21.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:21.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:21.188 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:48:21.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:48:21.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:21.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:48:21.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:21.448 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:21.441+0000 7f4472953a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:48:22.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:22.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:22.401 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:48:22.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:48:22.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:22.402 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:48:22.611 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2465845791,v1:127.0.0.1:6801/2465845791] [v2:127.0.0.1:6802/2465845791,v1:127.0.0.1:6803/2465845791] exists,up 982d1365-09e1-467c-bc52-574fe278ae5c 2026-02-17T04:48:22.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:248: TEST_0_osd: run_osd td/ok-to-stop 1 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/1 2026-02-17T04:48:22.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/1' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/1/journal' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:22.616 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T04:48:22.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/1 2026-02-17T04:48:22.618 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:48:22.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=61ad2fbe-e64e-433c-b0b0-fc7e101499c0 2026-02-17T04:48:22.619 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 61ad2fbe-e64e-433c-b0b0-fc7e101499c0 2026-02-17T04:48:22.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 61ad2fbe-e64e-433c-b0b0-fc7e101499c0' 2026-02-17T04:48:22.620 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:48:22.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAW85Npz2w9JRAANUBObLF/G+lH4imYQnnJHg== 2026-02-17T04:48:22.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAW85Npz2w9JRAANUBObLF/G+lH4imYQnnJHg=="}' 2026-02-17T04:48:22.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 61ad2fbe-e64e-433c-b0b0-fc7e101499c0 -i td/ok-to-stop/1/new.json 2026-02-17T04:48:22.840 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:48:22.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/1/new.json 2026-02-17T04:48:22.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAW85Npz2w9JRAANUBObLF/G+lH4imYQnnJHg== --osd-uuid 61ad2fbe-e64e-433c-b0b0-fc7e101499c0 2026-02-17T04:48:22.866 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:22.861+0000 7fcd61307a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:22.867 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:22.861+0000 7fcd61307a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:22.868 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:22.861+0000 7fcd61307a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:22.869 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:22.865+0000 7fcd61307a40 -1 bdev(0x562837cce400 td/ok-to-stop/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:48:22.869 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:22.865+0000 7fcd61307a40 -1 bluestore(td/ok-to-stop/1) _read_fsid unparsable uuid 2026-02-17T04:48:23.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/1/keyring 2026-02-17T04:48:23.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:48:23.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:48:23.509 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:48:23.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:48:23.788 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:48:23.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:48:23.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:48:23.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:48:23.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:48:23.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:48:23.804 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:23.797+0000 7f208e4b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:23.806 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:23.801+0000 7f208e4b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:23.806 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:23.801+0000 7f208e4b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:24.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:48:24.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:48:24.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:48:24.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:48:24.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:48:24.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:24.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:48:24.010 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:24.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:24.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:48:24.174 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:24.169+0000 7f208e4b5a40 -1 Falling back to public interface 2026-02-17T04:48:24.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:24.824 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:24.817+0000 7f208e4b5a40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:48:25.226 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:25.226 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:25.226 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:48:25.227 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:48:25.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:25.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:48:25.452 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3701036700,v1:127.0.0.1:6811/3701036700] [v2:127.0.0.1:6812/3701036700,v1:127.0.0.1:6813/3701036700] exists,up 61ad2fbe-e64e-433c-b0b0-fc7e101499c0 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:249: TEST_0_osd: run_osd td/ok-to-stop 2 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:48:25.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/2 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/2' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/2/journal' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:25.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:48:25.458 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:48:25.459 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:48:25.459 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:48:25.459 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:48:25.459 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/2 2026-02-17T04:48:25.459 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:48:25.460 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d98b4a41-baa4-40c8-9bea-d1c4e09a2e6a 2026-02-17T04:48:25.460 INFO:tasks.workunit.client.0.trial105.stdout:add osd2 d98b4a41-baa4-40c8-9bea-d1c4e09a2e6a 2026-02-17T04:48:25.460 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d98b4a41-baa4-40c8-9bea-d1c4e09a2e6a' 2026-02-17T04:48:25.460 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:48:25.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAZ85NptbbbGxAAuZzGa4tNkVaXh1VX4lhovQ== 2026-02-17T04:48:25.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAZ85NptbbbGxAAuZzGa4tNkVaXh1VX4lhovQ=="}' 2026-02-17T04:48:25.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d98b4a41-baa4-40c8-9bea-d1c4e09a2e6a -i td/ok-to-stop/2/new.json 2026-02-17T04:48:25.690 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:48:25.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/2/new.json 2026-02-17T04:48:25.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAZ85NptbbbGxAAuZzGa4tNkVaXh1VX4lhovQ== --osd-uuid d98b4a41-baa4-40c8-9bea-d1c4e09a2e6a 2026-02-17T04:48:25.723 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:25.717+0000 7f341ea11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:25.724 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:25.717+0000 7f341ea11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:25.725 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:25.721+0000 7f341ea11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:25.726 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:25.721+0000 7f341ea11a40 -1 bdev(0x55be40ed0400 td/ok-to-stop/2/block) open stat got: (1) Operation not permitted 2026-02-17T04:48:25.726 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:25.721+0000 7f341ea11a40 -1 bluestore(td/ok-to-stop/2) _read_fsid unparsable uuid 2026-02-17T04:48:26.367 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/2/keyring 2026-02-17T04:48:26.367 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:48:26.368 INFO:tasks.workunit.client.0.trial105.stdout:adding osd2 key to auth repository 2026-02-17T04:48:26.368 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T04:48:26.368 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:48:26.656 INFO:tasks.workunit.client.0.trial105.stdout:start osd.2 2026-02-17T04:48:26.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T04:48:26.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:48:26.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:48:26.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:48:26.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:48:26.678 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:26.673+0000 7f46d39d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:26.680 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:26.673+0000 7f46d39d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:26.681 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:26.677+0000 7f46d39d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:26.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T04:48:26.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:48:26.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T04:48:26.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:48:26.872 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:48:26.872 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:26.872 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:26.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:48:26.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:26.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:48:27.078 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:27.073+0000 7f46d39d3a40 -1 Falling back to public interface 2026-02-17T04:48:27.085 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:27.626 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:27.621+0000 7f46d39d3a40 -1 osd.2 0 log_to_monitors true 2026-02-17T04:48:28.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:28.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:28.087 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:48:28.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:48:28.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:28.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:48:28.294 INFO:tasks.workunit.client.0.trial105.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/201370908,v1:127.0.0.1:6819/201370908] [v2:127.0.0.1:6820/201370908,v1:127.0.0.1:6821/201370908] exists,up d98b4a41-baa4-40c8-9bea-d1c4e09a2e6a 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:250: TEST_0_osd: run_osd td/ok-to-stop 3 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/3 2026-02-17T04:48:28.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/3' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/3/journal' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:28.299 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:48:28.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:48:28.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:48:28.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:48:28.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:48:28.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/3 2026-02-17T04:48:28.301 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:48:28.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=46f85c27-ab19-49b0-8357-70acfd069026 2026-02-17T04:48:28.302 INFO:tasks.workunit.client.0.trial105.stdout:add osd3 46f85c27-ab19-49b0-8357-70acfd069026 2026-02-17T04:48:28.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 46f85c27-ab19-49b0-8357-70acfd069026' 2026-02-17T04:48:28.302 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:48:28.311 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAc85Np3aR1EhAAiT5ynGkhYcBRUxsYnT5Ocg== 2026-02-17T04:48:28.311 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAc85Np3aR1EhAAiT5ynGkhYcBRUxsYnT5Ocg=="}' 2026-02-17T04:48:28.311 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 46f85c27-ab19-49b0-8357-70acfd069026 -i td/ok-to-stop/3/new.json 2026-02-17T04:48:28.517 INFO:tasks.workunit.client.0.trial105.stdout:3 2026-02-17T04:48:28.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/3/new.json 2026-02-17T04:48:28.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAc85Np3aR1EhAAiT5ynGkhYcBRUxsYnT5Ocg== --osd-uuid 46f85c27-ab19-49b0-8357-70acfd069026 2026-02-17T04:48:28.542 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:28.537+0000 7f51471eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:28.544 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:28.537+0000 7f51471eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:28.545 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:28.541+0000 7f51471eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:28.546 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:28.541+0000 7f51471eda40 -1 bdev(0x5628b52b6400 td/ok-to-stop/3/block) open stat got: (1) Operation not permitted 2026-02-17T04:48:28.546 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:28.541+0000 7f51471eda40 -1 bluestore(td/ok-to-stop/3) _read_fsid unparsable uuid 2026-02-17T04:48:29.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/3/keyring 2026-02-17T04:48:29.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:48:29.172 INFO:tasks.workunit.client.0.trial105.stdout:adding osd3 key to auth repository 2026-02-17T04:48:29.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-17T04:48:29.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:48:29.460 INFO:tasks.workunit.client.0.trial105.stdout:start osd.3 2026-02-17T04:48:29.461 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-17T04:48:29.461 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:48:29.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:48:29.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:48:29.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:48:29.484 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:29.477+0000 7f510bf0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:29.486 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:29.481+0000 7f510bf0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:29.487 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:29.481+0000 7f510bf0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:29.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:48:29.676 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:48:29.676 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:29.676 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-17T04:48:29.888 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:48:29.918 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:29.913+0000 7f510bf0fa40 -1 Falling back to public interface 2026-02-17T04:48:30.494 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:48:30.489+0000 7f510bf0fa40 -1 osd.3 0 log_to_monitors true 2026-02-17T04:48:30.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:48:30.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:48:30.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:48:30.889 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:48:30.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:48:30.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-17T04:48:31.110 INFO:tasks.workunit.client.0.trial105.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3500485684,v1:127.0.0.1:6827/3500485684] [v2:127.0.0.1:6828/3500485684,v1:127.0.0.1:6829/3500485684] exists,up 46f85c27-ab19-49b0-8357-70acfd069026 2026-02-17T04:48:31.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:48:31.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:48:31.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:48:31.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:252: TEST_0_osd: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-17T04:48:31.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:253: TEST_0_osd: ceph osd pool create ec erasure ec-profile 2026-02-17T04:48:31.723 INFO:tasks.workunit.client.0.trial105.stderr:pool 'ec' already exists 2026-02-17T04:48:31.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:255: TEST_0_osd: wait_for_clean 2026-02-17T04:48:31.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:48:31.736 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:48:31.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:48:31.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:48:31.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:48:31.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:48:31.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:48:31.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:48:31.842 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:48:32.058 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:48:32.059 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:48:32.059 INFO:tasks.workunit.client.0.trial105.stderr:2 2026-02-17T04:48:32.059 INFO:tasks.workunit.client.0.trial105.stderr:3' 2026-02-17T04:48:32.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:48:32.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:32.059 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:48:32.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-17T04:48:32.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-17T04:48:32.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T04:48:32.138 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:32.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:48:32.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-17T04:48:32.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-17T04:48:32.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668' 2026-02-17T04:48:32.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:32.218 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:48:32.296 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-17T04:48:32.296 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-17T04:48:32.296 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-55834574851' 2026-02-17T04:48:32.296 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:32.297 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T04:48:32.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-17T04:48:32.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-17T04:48:32.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-55834574851 3-77309411331' 2026-02-17T04:48:32.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:32.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-17T04:48:32.378 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:32.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:48:32.379 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-17T04:48:32.379 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:32.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-17T04:48:32.380 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836484 2026-02-17T04:48:32.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T04:48:32.381 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:48:32.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T04:48:32.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:48:33.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:48:33.601 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:48:33.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-17T04:48:33.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:33.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705668 2026-02-17T04:48:33.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:33.812 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:48:33.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-17T04:48:33.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:33.814 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705668 2026-02-17T04:48:33.814 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705668 2026-02-17T04:48:33.814 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T04:48:33.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:48:34.037 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T04:48:34.037 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:34.038 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574851 2026-02-17T04:48:34.038 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:34.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:48:34.039 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-17T04:48:34.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:34.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574851 2026-02-17T04:48:34.041 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574851 2026-02-17T04:48:34.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T04:48:34.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:48:34.258 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-17T04:48:34.258 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:34.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411331 2026-02-17T04:48:34.259 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:34.260 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T04:48:34.260 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411331 2026-02-17T04:48:34.260 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:34.262 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-17T04:48:34.262 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.3 seq 77309411331 2026-02-17T04:48:34.262 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411331' 2026-02-17T04:48:34.262 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T04:48:34.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411331 -lt 77309411331 2026-02-17T04:48:34.489 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:48:34.489 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:34.490 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:34.776 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-17T04:48:34.776 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:48:34.776 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:48:34.777 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:48:34.777 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:48:34.777 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:48:34.777 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:48:34.777 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:48:34.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-17T04:48:34.992 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:48:34.992 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:34.992 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:35.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-17T04:48:35.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:48:35.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:48:35.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:258: TEST_0_osd: ceph osd pool set ec min_size 3 2026-02-17T04:48:35.579 INFO:tasks.workunit.client.0.trial105.stderr:set pool 1 min_size to 3 2026-02-17T04:48:35.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:259: TEST_0_osd: wait_for_clean 2026-02-17T04:48:35.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:48:35.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:48:35.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:48:35.592 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:48:35.593 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:48:35.593 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:48:35.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:48:35.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:48:35.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:48:35.691 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:48:35.692 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:48:35.692 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:48:35.692 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:48:35.692 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:48:35.692 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:48:35.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:48:35.914 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:48:35.914 INFO:tasks.workunit.client.0.trial105.stderr:2 2026-02-17T04:48:35.914 INFO:tasks.workunit.client.0.trial105.stderr:3' 2026-02-17T04:48:35.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:48:35.915 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:35.915 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:48:35.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-17T04:48:35.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-17T04:48:35.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T04:48:35.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:35.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:48:36.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-17T04:48:36.075 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-17T04:48:36.075 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-38654705671' 2026-02-17T04:48:36.075 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:36.075 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:48:36.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-17T04:48:36.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-17T04:48:36.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-38654705671 2-55834574854' 2026-02-17T04:48:36.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:36.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T04:48:36.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-17T04:48:36.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-17T04:48:36.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-38654705671 2-55834574854 3-77309411333' 2026-02-17T04:48:36.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:36.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836487 2026-02-17T04:48:36.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:36.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:48:36.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-17T04:48:36.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:36.233 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836487 2026-02-17T04:48:36.234 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836487 2026-02-17T04:48:36.234 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T04:48:36.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:48:36.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-17T04:48:36.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:48:37.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:48:37.454 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:48:37.676 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T04:48:37.676 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:37.677 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705671 2026-02-17T04:48:37.677 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:37.677 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:48:37.678 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-17T04:48:37.678 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:37.680 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705671 2026-02-17T04:48:37.680 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705671 2026-02-17T04:48:37.680 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T04:48:37.681 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:48:37.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T04:48:37.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:37.907 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574854 2026-02-17T04:48:37.908 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:37.908 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:48:37.909 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574854 2026-02-17T04:48:37.909 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:37.911 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-17T04:48:37.911 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574854 2026-02-17T04:48:37.911 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-17T04:48:37.912 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:48:38.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-17T04:48:38.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:38.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411333 2026-02-17T04:48:38.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:38.131 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T04:48:38.132 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411333 2026-02-17T04:48:38.132 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:38.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-17T04:48:38.133 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.3 seq 77309411333 2026-02-17T04:48:38.134 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411333' 2026-02-17T04:48:38.134 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T04:48:38.357 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411333 2026-02-17T04:48:38.357 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:48:38.358 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:38.358 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:48:38.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:48:38.647 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:48:38.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-17T04:48:38.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:48:38.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:38.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:39.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-17T04:48:39.138 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:48:39.138 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:48:39.138 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:261: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-17T04:48:39.346 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:39.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:262: TEST_0_osd: ceph osd ok-to-stop 1 2026-02-17T04:48:39.568 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:39.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:263: TEST_0_osd: ceph osd ok-to-stop 2 2026-02-17T04:48:39.785 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:39.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:264: TEST_0_osd: ceph osd ok-to-stop 3 2026-02-17T04:48:40.018 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:40.027 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:265: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-17T04:48:40.196 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":false,"osds":[0,1],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:40.196 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T04:48:40.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:266: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-17T04:48:40.354 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":false,"osds":[2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:40.354 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T04:48:40.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-17T04:48:40.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: grep '[0]' 2026-02-17T04:48:40.574 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:40.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: ceph osd ok-to-stop 1 --max 2 2026-02-17T04:48:40.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: grep '[1]' 2026-02-17T04:48:40.786 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:40.786 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:271: TEST_0_osd: ceph osd pool set ec min_size 2 2026-02-17T04:48:41.094 INFO:tasks.workunit.client.0.trial105.stderr:set pool 1 min_size to 2 2026-02-17T04:48:41.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:272: TEST_0_osd: wait_for_clean 2026-02-17T04:48:41.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:48:41.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:48:41.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:48:41.110 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:48:41.111 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:48:41.111 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:48:41.112 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:48:41.112 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:48:41.112 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:48:41.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:48:41.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:48:41.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:48:41.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:48:41.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:48:41.219 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr:2 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr:3' 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:41.434 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:48:41.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-17T04:48:41.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-17T04:48:41.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T04:48:41.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:41.513 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:48:41.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-17T04:48:41.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-17T04:48:41.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705674' 2026-02-17T04:48:41.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:41.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:48:41.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-17T04:48:41.674 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-17T04:48:41.674 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857' 2026-02-17T04:48:41.674 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:41.674 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T04:48:41.750 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-17T04:48:41.750 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-17T04:48:41.750 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411337' 2026-02-17T04:48:41.750 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:41.751 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-17T04:48:41.751 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:41.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:48:41.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-17T04:48:41.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:41.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-17T04:48:41.753 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836490 2026-02-17T04:48:41.754 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T04:48:41.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:48:41.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-17T04:48:41.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:48:42.978 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:48:42.978 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:48:43.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-17T04:48:43.197 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:43.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705674 2026-02-17T04:48:43.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:43.198 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:48:43.199 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-17T04:48:43.199 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:43.201 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705674 2026-02-17T04:48:43.201 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705674 2026-02-17T04:48:43.201 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T04:48:43.201 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:48:43.411 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T04:48:43.412 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:43.412 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574857 2026-02-17T04:48:43.412 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:43.413 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:48:43.414 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574857 2026-02-17T04:48:43.414 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:43.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-17T04:48:43.416 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574857 2026-02-17T04:48:43.416 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T04:48:43.416 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:48:43.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-17T04:48:43.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:43.632 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411337 2026-02-17T04:48:43.632 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:43.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T04:48:43.634 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411337 2026-02-17T04:48:43.634 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:43.635 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-17T04:48:43.636 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.3 seq 77309411337 2026-02-17T04:48:43.636 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411337' 2026-02-17T04:48:43.636 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T04:48:43.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-17T04:48:43.850 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:48:43.851 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:43.851 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:48:44.129 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:48:44.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:48:44.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-17T04:48:44.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:48:44.339 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:44.339 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:44.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-17T04:48:44.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:48:44.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:48:44.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:274: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-17T04:48:44.826 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:44.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:275: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-17T04:48:45.042 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[2,3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:45.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:276: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-17T04:48:45.219 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:45.220 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T04:48:45.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:277: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-17T04:48:45.376 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:45.377 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T04:48:45.382 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-17T04:48:45.382 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: grep '[0,1]' 2026-02-17T04:48:45.595 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:45.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: ceph osd ok-to-stop 0 --max 20 2026-02-17T04:48:45.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: grep '[0,1]' 2026-02-17T04:48:45.811 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:45.812 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: ceph osd ok-to-stop 2 --max 2 2026-02-17T04:48:45.812 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: grep '[2,3]' 2026-02-17T04:48:46.032 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:46.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: ceph osd ok-to-stop 2 --max 20 2026-02-17T04:48:46.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: grep '[2,3]' 2026-02-17T04:48:46.252 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:46.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:285: TEST_0_osd: kill_daemons td/ok-to-stop TERM osd.0 2026-02-17T04:48:46.253 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:46.253 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:46.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:46.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:46.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:46.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:46.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:286: TEST_0_osd: ceph osd down 0 2026-02-17T04:48:46.769 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 is already down. 2026-02-17T04:48:46.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:287: TEST_0_osd: wait_for_peered 2026-02-17T04:48:46.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-17T04:48:46.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-17T04:48:46.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-17T04:48:46.779 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-17T04:48:46.780 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:48:46.780 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:48:46.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:48:46.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:48:46.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:48:46.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:48:46.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-17T04:48:46.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-17T04:48:46.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-17T04:48:46.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:48:46.886 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr:2 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr:3' 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:47.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:48:47.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-17T04:48:47.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-17T04:48:47.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-17T04:48:47.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-17T04:48:47.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:47.159 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:48:47.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705677 2026-02-17T04:48:47.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705677 2026-02-17T04:48:47.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-38654705677' 2026-02-17T04:48:47.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:47.238 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T04:48:47.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574860 2026-02-17T04:48:47.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574860 2026-02-17T04:48:47.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-38654705677 2-55834574860' 2026-02-17T04:48:47.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:48:47.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T04:48:47.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411340 2026-02-17T04:48:47.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411340 2026-02-17T04:48:47.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-38654705677 2-55834574860 3-77309411340' 2026-02-17T04:48:47.400 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:47.401 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705677 2026-02-17T04:48:47.401 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:47.401 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:48:47.402 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705677 2026-02-17T04:48:47.402 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:47.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705677 2026-02-17T04:48:47.403 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 38654705677 2026-02-17T04:48:47.404 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-17T04:48:47.404 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:48:47.617 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705675 -lt 38654705677 2026-02-17T04:48:47.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:48:48.618 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:48:48.619 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:48:48.840 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705675 -lt 38654705677 2026-02-17T04:48:48.840 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:48:49.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T04:48:49.841 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:48:50.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705677 -lt 38654705677 2026-02-17T04:48:50.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:50.056 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574860 2026-02-17T04:48:50.056 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:50.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T04:48:50.057 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574860 2026-02-17T04:48:50.057 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:50.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574860 2026-02-17T04:48:50.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574860' 2026-02-17T04:48:50.059 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.2 seq 55834574860 2026-02-17T04:48:50.059 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T04:48:50.266 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574861 -lt 55834574860 2026-02-17T04:48:50.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:48:50.267 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411340 2026-02-17T04:48:50.268 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:48:50.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T04:48:50.268 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411340 2026-02-17T04:48:50.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:48:50.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411340 2026-02-17T04:48:50.270 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.3 seq 77309411340 2026-02-17T04:48:50.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411340' 2026-02-17T04:48:50.270 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T04:48:50.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411340 -lt 77309411340 2026-02-17T04:48:50.480 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-17T04:48:50.481 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:50.481 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 32 == 0 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-17T04:48:50.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-17T04:48:50.765 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-17T04:48:50.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=32 2026-02-17T04:48:50.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-17T04:48:50.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:48:50.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:48:51.274 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 32 = 32 2026-02-17T04:48:51.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-17T04:48:51.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-17T04:48:51.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:289: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-17T04:48:51.473 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":0,"num_not_ok_pgs":0} 2026-02-17T04:48:51.483 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:290: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-17T04:48:51.695 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:51.703 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:291: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-17T04:48:51.867 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:51.868 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T04:48:51.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:292: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-17T04:48:52.022 INFO:tasks.workunit.client.0.trial105.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T04:48:52.022 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T04:48:52.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:52.027 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:52.028 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:52.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:52.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:52.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:52.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:52.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:52.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:52.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:52.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:52.346 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:52.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:52.347 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:52.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:52.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:52.349 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:52.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:52.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:52.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:52.350 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:52.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:52.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:52.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:52.370 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:52.370 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:52.370 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:52.370 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:52.372 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:52.373 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:52.374 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:52.374 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:52.374 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:52.374 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:52.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:52.375 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:52.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:52.376 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:52.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:52.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:52.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:52.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:52.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:52.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:52.379 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:52.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:52.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:52.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:52.382 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:52.383 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:52.383 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:52.383 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:52.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:52.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:52.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:48:52.386 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:48:52.386 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:52.386 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:52.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:48:52.387 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_1_mon_checks td/ok-to-stop 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:28: TEST_1_mon_checks: local dir=td/ok-to-stop 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:30: TEST_1_mon_checks: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:32: TEST_1_mon_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:48:52.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:52.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:48:52.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:48:52.409 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:52.410 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:52.410 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:52.411 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:52.411 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:52.411 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:52.411 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:52.411 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:48:52.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:52.440 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:52.441 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:52.441 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:52.442 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:52.442 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:52.442 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:52.442 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:48:52.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:48:52.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:52.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:52.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:52.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:52.496 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:52.497 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:52.497 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:48:52.551 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:34: TEST_1_mon_checks: ceph mon ok-to-stop dne 2026-02-17T04:48:52.653 INFO:tasks.workunit.client.0.trial105.stderr:invalid mon(s) specified: [dne] 2026-02-17T04:48:52.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:35: TEST_1_mon_checks: ceph mon ok-to-stop a 2026-02-17T04:48:52.748 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a] 2026-02-17T04:48:52.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:37: TEST_1_mon_checks: ceph mon ok-to-add-offline 2026-02-17T04:48:52.837 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-17T04:48:52.840 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:39: TEST_1_mon_checks: ceph mon ok-to-rm a 2026-02-17T04:48:52.924 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-17T04:48:52.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:40: TEST_1_mon_checks: ceph mon ok-to-rm dne 2026-02-17T04:48:53.031 INFO:tasks.workunit.client.0.trial105.stderr:mon.dne does not exist 2026-02-17T04:48:53.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:53.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:53.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:53.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:53.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:53.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:53.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:53.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:48:53.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:53.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:53.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:53.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:53.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:53.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:53.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:53.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:53.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:53.153 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:53.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:53.155 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:53.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:53.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:53.157 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:53.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:53.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:53.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:53.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:53.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:53.160 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:53.160 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:53.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:53.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:53.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:53.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:53.166 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:53.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:53.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:53.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:53.169 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:53.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:53.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:53.171 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:53.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:53.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:53.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:53.173 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:53.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:53.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:53.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:53.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:53.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:53.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:53.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:53.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:48:53.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:48:53.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:48:53.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_2_mons_checks td/ok-to-stop 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:44: TEST_2_mons_checks: local dir=td/ok-to-stop 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:46: TEST_2_mons_checks: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151 ' 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:48: TEST_2_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:48:53.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:53.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:53.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:48:53.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:53.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:48:53.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:48:53.203 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:53.203 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:53.203 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:53.203 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:53.203 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.203 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.204 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:53.204 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:48:53.232 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:53.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:48:53.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:53.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:53.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:53.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:53.234 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:53.234 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:53.234 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:53.234 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:53.234 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.234 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.235 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:53.235 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:53.235 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:48:53.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:48:53.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:53.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:53.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:53.291 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:53.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:53.293 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:48:53.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:49: TEST_2_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T04:48:53.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:53.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:53.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T04:48:53.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:53.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T04:48:53.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T04:48:53.369 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:53.369 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:53.369 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:53.369 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:53.369 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.370 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.370 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:53.370 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T04:48:53.396 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:53.397 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T04:48:53.397 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:53.397 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:48:53.398 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.399 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:48:53.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:53.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get fsid 2026-02-17T04:48:53.456 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T04:48:53.456 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:53.457 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:48:53.458 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:53.458 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get mon_host 2026-02-17T04:48:53.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:51: TEST_2_mons_checks: ceph mon ok-to-stop dne 2026-02-17T04:48:53.625 INFO:tasks.workunit.client.0.trial105.stderr:invalid mon(s) specified: [dne] 2026-02-17T04:48:53.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:52: TEST_2_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:48:53.733 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (b) after stopping mons [a] 2026-02-17T04:48:53.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:53: TEST_2_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:48:53.822 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b] 2026-02-17T04:48:53.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:54: TEST_2_mons_checks: ceph mon ok-to-stop a b 2026-02-17T04:48:53.915 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b] 2026-02-17T04:48:53.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:56: TEST_2_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:48:54.021 INFO:tasks.workunit.client.0.trial105.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T04:48:54.032 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:58: TEST_2_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:48:54.141 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:48:54.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:59: TEST_2_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:48:54.265 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:48:54.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:60: TEST_2_mons_checks: ceph mon ok-to-rm dne 2026-02-17T04:48:54.381 INFO:tasks.workunit.client.0.trial105.stderr:mon.dne does not exist 2026-02-17T04:48:54.389 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:54.390 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:54.390 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:54.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:54.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:54.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:54.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:54.499 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:54.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:54.504 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:54.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:54.506 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:54.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:54.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:54.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:54.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:54.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:54.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:54.509 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:54.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:54.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:54.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:54.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:54.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.515 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:54.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:54.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:54.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:54.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:54.519 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:54.520 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:48:54.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:48:54.521 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:48:54.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:48:54.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:48:54.522 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:48:54.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:54.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:48:54.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:48:54.524 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:48:54.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:48:54.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:48:54.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:48:54.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:48:54.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:48:54.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:48:54.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:48:54.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:48:54.531 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:48:54.531 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.531 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.531 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:48:54.533 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_3_mons_checks td/ok-to-stop 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:64: TEST_3_mons_checks: local dir=td/ok-to-stop 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:66: TEST_3_mons_checks: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152 ' 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:68: TEST_3_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:54.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:48:54.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:54.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:48:54.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:48:54.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:54.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:54.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:54.556 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:54.556 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.556 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.556 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:54.556 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:48:54.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:54.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:48:54.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:54.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:54.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:54.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:54.585 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:54.586 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:54.586 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:54.586 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:54.586 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.586 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.587 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:54.587 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:54.587 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:48:54.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:48:54.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:54.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:48:54.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:54.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:54.645 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:48:54.645 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:48:54.645 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:48:54.646 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:54.646 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.646 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.646 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:48:54.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:54.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:48:54.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:69: TEST_3_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T04:48:54.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:54.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:54.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T04:48:54.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:54.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T04:48:54.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.723 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:54.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T04:48:54.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:54.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T04:48:54.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:54.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:48:54.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:54.755 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:54.755 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:48:54.755 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:48:54.755 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:48:54.755 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:54.756 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.756 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.756 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:48:54.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:54.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get fsid 2026-02-17T04:48:54.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T04:48:54.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:54.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:48:54.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:54.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:54.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get mon_host 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:70: TEST_3_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-17T04:48:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:48:54.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-17T04:48:54.918 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:48:54.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-17T04:48:54.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:54.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T04:48:54.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:48:54.921 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:48:54.921 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.c.asok 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:54.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.c.asok config get fsid 2026-02-17T04:48:54.977 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-17T04:48:54.977 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:48:54.977 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T04:48:54.977 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:48:54.978 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:48:54.978 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T04:48:54.978 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T04:48:54.978 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T04:48:54.979 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:48:54.979 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:48:54.979 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:48:54.979 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.c.asok 2026-02-17T04:48:54.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:48:54.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.c.asok config get mon_host 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:71: TEST_3_mons_checks: wait_for_quorum 60 3 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:48:55.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:48:55.034 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303795 2026-02-17T04:48:55.035 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:48:55.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303735 -lt 1771303795 ]] 2026-02-17T04:48:55.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:48:55.037 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:48:55.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:48:55.355 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:48:55.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:48:55.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:48:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:48:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:48:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:48:55.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:48:55.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:48:55.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:73: TEST_3_mons_checks: ceph mon ok-to-stop dne 2026-02-17T04:48:58.472 INFO:tasks.workunit.client.0.trial105.stderr:invalid mon(s) specified: [dne] 2026-02-17T04:48:58.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:74: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:48:58.593 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c) after stopping [a] 2026-02-17T04:48:58.603 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:75: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:48:58.707 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-17T04:48:58.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:76: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:48:58.822 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b) after stopping [c] 2026-02-17T04:48:58.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:77: TEST_3_mons_checks: ceph mon ok-to-stop a b 2026-02-17T04:48:58.919 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a,b] 2026-02-17T04:48:58.921 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:78: TEST_3_mons_checks: ceph mon ok-to-stop b c 2026-02-17T04:48:59.009 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b,c] 2026-02-17T04:48:59.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:79: TEST_3_mons_checks: ceph mon ok-to-stop a b c 2026-02-17T04:48:59.096 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b,c] 2026-02-17T04:48:59.098 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:81: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:48:59.204 INFO:tasks.workunit.client.0.trial105.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T04:48:59.212 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:83: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:48:59.319 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:48:59.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:84: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:48:59.445 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:48:59.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:85: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:48:59.571 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.c 2026-02-17T04:48:59.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:87: TEST_3_mons_checks: kill_daemons td/ok-to-stop KILL mon.b 2026-02-17T04:48:59.579 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:48:59.579 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:48:59.580 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:48:59.580 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:48:59.580 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:48:59.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:48:59.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:88: TEST_3_mons_checks: wait_for_quorum 60 2 2026-02-17T04:48:59.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:48:59.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=2 2026-02-17T04:48:59.688 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:48:59.688 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 2 ]] 2026-02-17T04:48:59.688 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:48:59.688 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:48:59.688 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303799 2026-02-17T04:48:59.689 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:48:59.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303739 -lt 1771303799 ]] 2026-02-17T04:48:59.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:48:59.689 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:48:59.798 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:48:59.798 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:48:59.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:48:59.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:48:59.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:48:59.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:48:59.809 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:48:59.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303739 -lt 1771303799 ]] 2026-02-17T04:48:59.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:48:59.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:48:59.927 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:48:59.927 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:48:59.928 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:48:59.928 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:48:59.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:48:59.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:48:59.937 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:48:59.939 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303739 -lt 1771303799 ]] 2026-02-17T04:48:59.939 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:48:59.939 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.051 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.062 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.063 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.183 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.184 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.184 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.195 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.195 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.196 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.307 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.307 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.308 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.308 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.317 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.319 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.435 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.435 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.436 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.436 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.446 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.447 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.563 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.564 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.564 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.564 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.575 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.576 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.577 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.577 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.694 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.694 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.695 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.695 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.705 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.706 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.706 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.706 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.821 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.832 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.833 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:00.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:00.949 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:00.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:00.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:00.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:00.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:00.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303740 -lt 1771303799 ]] 2026-02-17T04:49:00.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:00.961 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:04.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:04.078 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:04.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:04.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:04.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:04.089 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303744 -lt 1771303799 ]] 2026-02-17T04:49:04.089 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:04.090 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:04.200 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:04.201 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.201 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:04.201 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.211 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:04.212 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:04.212 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:04.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303744 -lt 1771303799 ]] 2026-02-17T04:49:04.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:04.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:04.328 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:04.329 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.329 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.329 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:04.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:04.340 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:04.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:04.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303744 -lt 1771303799 ]] 2026-02-17T04:49:04.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:04.342 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:04.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:04.456 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:04.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:04.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:04.468 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:04.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303744 -lt 1771303799 ]] 2026-02-17T04:49:04.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:04.469 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:04.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:04.585 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:04.585 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.596 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:04.596 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:04.596 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:04.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303744 -lt 1771303799 ]] 2026-02-17T04:49:04.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:04.597 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:04.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:04.709 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.710 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:04.710 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:04.720 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:04.720 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:04.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:04.721 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303744 -lt 1771303799 ]] 2026-02-17T04:49:04.721 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:04.722 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:07.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:07.838 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:07.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:07.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:07.848 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:07.848 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:07.849 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:07.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303747 -lt 1771303799 ]] 2026-02-17T04:49:07.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:07.850 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:07.962 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:07.963 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:07.964 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:07.964 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:07.973 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:07.973 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:07.973 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:07.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303747 -lt 1771303799 ]] 2026-02-17T04:49:07.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:07.975 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:08.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:08.088 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.088 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:08.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:08.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:08.099 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:08.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303748 -lt 1771303799 ]] 2026-02-17T04:49:08.101 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:08.101 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:08.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:08.216 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.217 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:08.217 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:08.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:08.228 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:08.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303748 -lt 1771303799 ]] 2026-02-17T04:49:08.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:08.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:08.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:08.343 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.343 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:08.343 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:08.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:08.353 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:08.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303748 -lt 1771303799 ]] 2026-02-17T04:49:08.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:08.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:08.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:08.465 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.465 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:08.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:08.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:08.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:08.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303748 -lt 1771303799 ]] 2026-02-17T04:49:08.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:08.478 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:08.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:08.594 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.595 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:08.595 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:08.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:08.604 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:08.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303748 -lt 1771303799 ]] 2026-02-17T04:49:08.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:08.606 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:08.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:08.716 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.717 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:08.717 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:08.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:08.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:08.728 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:08.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303748 -lt 1771303799 ]] 2026-02-17T04:49:08.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T04:49:08.729 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:17.843 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:17.844 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:17.844 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T04:49:17.845 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:48:54.548028Z","created":"2026-02-17T04:48:54.548028Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548025+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548026+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:48:54:548027+0000","crush_location":[]}]}}' 2026-02-17T04:49:17.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:49:17.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:49:17.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:49:17.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:49:17.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:49:17.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:90: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:49:17.940 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a] 2026-02-17T04:49:17.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:91: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:49:18.048 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-17T04:49:18.056 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:92: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:49:21.178 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [c] 2026-02-17T04:49:21.180 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:94: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:49:21.269 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-17T04:49:21.272 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:96: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:49:24.361 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-17T04:49:24.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:97: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:49:24.469 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:49:24.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:98: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:49:24.577 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-17T04:49:24.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:49:24.580 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:24.580 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:24.580 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:24.580 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:24.580 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:24.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:24.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:24.694 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:24.695 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:49:24.695 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:49:24.696 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:49:24.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:49:24.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:49:24.697 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:49:24.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:24.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:49:24.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:49:24.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:24.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:49:24.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:49:24.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:49:24.706 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:49:24.706 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.706 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.706 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:24.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:24.709 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:24.709 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:24.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:24.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:49:24.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:49:24.712 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:49:24.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:49:24.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:49:24.713 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:49:24.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:24.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:49:24.715 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:49:24.715 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:24.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:49:24.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:49:24.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:49:24.718 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:49:24.718 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.718 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.718 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:49:24.719 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:49:24.719 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:49:24.720 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:49:24.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:49:24.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.721 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:49:24.722 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:49:24.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_4_mons_checks td/ok-to-stop 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:102: TEST_4_mons_checks: local dir=td/ok-to-stop 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:104: TEST_4_mons_checks: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153 ' 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:106: TEST_4_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:49:24.724 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:49:24.744 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:24.744 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:24.744 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:24.744 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:24.744 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.744 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.745 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:24.745 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:49:24.777 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:24.778 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:49:24.778 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:24.778 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:49:24.778 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:24.778 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:24.780 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:49:24.837 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:49:24.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:24.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:49:24.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:24.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:24.838 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:49:24.838 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:49:24.839 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:49:24.839 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:24.839 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.839 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.839 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:49:24.839 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:24.840 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:107: TEST_4_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T04:49:24.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T04:49:24.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:24.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:24.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:24.919 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:24.919 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.919 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.919 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:24.919 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T04:49:24.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:24.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T04:49:24.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:24.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:49:24.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:24.949 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:49:24.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:24.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get fsid 2026-02-17T04:49:25.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T04:49:25.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:25.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:49:25.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:25.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:25.005 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:49:25.006 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:49:25.006 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:49:25.007 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:25.007 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.007 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.007 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:49:25.007 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:25.007 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get mon_host 2026-02-17T04:49:25.060 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:108: TEST_4_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-17T04:49:25.060 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:25.060 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:25.060 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-17T04:49:25.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:25.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-17T04:49:25.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-17T04:49:25.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:25.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:25.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:25.078 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:25.078 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.078 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.079 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:25.079 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-17T04:49:25.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:25.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-17T04:49:25.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:25.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T04:49:25.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:25.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:25.109 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T04:49:25.110 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T04:49:25.110 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T04:49:25.110 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:25.110 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.110 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.110 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.c.asok 2026-02-17T04:49:25.111 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:25.111 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.c.asok config get fsid 2026-02-17T04:49:25.167 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-17T04:49:25.167 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:25.167 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T04:49:25.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:25.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:25.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T04:49:25.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T04:49:25.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T04:49:25.169 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:25.169 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.169 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.170 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.c.asok 2026-02-17T04:49:25.170 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:25.170 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.c.asok config get mon_host 2026-02-17T04:49:25.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:109: TEST_4_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-17T04:49:25.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:25.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:25.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-17T04:49:25.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:25.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-17T04:49:25.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-17T04:49:25.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:25.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:25.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:25.244 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:25.244 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.244 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.244 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:25.244 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-17T04:49:25.271 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:25.272 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-17T04:49:25.272 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:25.273 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T04:49:25.273 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:25.274 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:25.274 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T04:49:25.274 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T04:49:25.274 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T04:49:25.274 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:25.275 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.275 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.275 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.d.asok 2026-02-17T04:49:25.275 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:25.275 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.d.asok config get fsid 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T04:49:25.330 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T04:49:25.331 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T04:49:25.332 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:25.332 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:25.332 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:25.332 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.d.asok 2026-02-17T04:49:25.332 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:25.332 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.d.asok config get mon_host 2026-02-17T04:49:25.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:110: TEST_4_mons_checks: wait_for_quorum 60 4 2026-02-17T04:49:25.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:49:25.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-17T04:49:25.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:49:25.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-17T04:49:25.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:49:25.386 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:49:25.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303825 2026-02-17T04:49:25.387 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:25.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303765 -lt 1771303825 ]] 2026-02-17T04:49:25.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:49:25.388 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:28.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:28.505 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:28.505 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:49:28.505 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:28.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:49:28.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:49:28.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:49:28.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:49:28.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:49:28.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:112: TEST_4_mons_checks: ceph mon ok-to-stop dne 2026-02-17T04:49:28.619 INFO:tasks.workunit.client.0.trial105.stderr:invalid mon(s) specified: [dne] 2026-02-17T04:49:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:113: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:49:28.746 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-17T04:49:28.756 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:114: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:49:28.859 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,c,d) after stopping [b] 2026-02-17T04:49:28.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:115: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:49:28.984 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b,d) after stopping [c] 2026-02-17T04:49:28.992 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:116: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-17T04:49:29.096 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b,c) after stopping [d] 2026-02-17T04:49:29.103 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:117: TEST_4_mons_checks: ceph mon ok-to-stop a b 2026-02-17T04:49:29.204 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [a,b] 2026-02-17T04:49:29.207 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:118: TEST_4_mons_checks: ceph mon ok-to-stop c d 2026-02-17T04:49:29.290 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (a,b) after stopping mons [c,d] 2026-02-17T04:49:29.292 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:120: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:49:29.391 INFO:tasks.workunit.client.0.trial105.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T04:49:29.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:122: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:49:29.519 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:49:29.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:123: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:49:29.636 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:49:29.646 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:124: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:49:29.754 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.c 2026-02-17T04:49:29.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:126: TEST_4_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-17T04:49:29.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:29.765 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:29.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:29.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:29.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:127: TEST_4_mons_checks: wait_for_quorum 60 3 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:49:29.873 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:49:29.874 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303829 2026-02-17T04:49:29.875 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:29.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303769 -lt 1771303829 ]] 2026-02-17T04:49:29.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:29.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:29.988 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:29.989 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:29.989 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:29.990 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:30.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:30.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:30.000 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:30.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303770 -lt 1771303829 ]] 2026-02-17T04:49:30.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:30.002 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:30.118 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:30.118 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:30.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:30.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:30.128 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:30.128 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:30.129 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:30.130 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303770 -lt 1771303829 ]] 2026-02-17T04:49:30.130 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:30.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:33.243 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:33.243 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:33.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:33.244 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:33.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:33.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:33.253 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:33.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303773 -lt 1771303829 ]] 2026-02-17T04:49:33.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:33.255 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:36.371 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:36.371 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:36.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:36.381 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:36.381 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:36.382 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303776 -lt 1771303829 ]] 2026-02-17T04:49:36.382 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:36.382 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:36.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:36.492 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.493 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:36.493 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:36.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:36.504 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:36.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303776 -lt 1771303829 ]] 2026-02-17T04:49:36.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:36.505 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:36.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:36.621 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.622 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:36.622 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:36.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:36.632 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:36.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303776 -lt 1771303829 ]] 2026-02-17T04:49:36.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:36.634 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:36.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:36.749 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.750 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:36.750 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:36.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:49:36.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:49:36.760 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:36.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303776 -lt 1771303829 ]] 2026-02-17T04:49:36.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:49:36.762 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:45.874 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:45.874 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:45.875 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:49:45.875 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:24.736797Z","created":"2026-02-17T04:49:24.736797Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736793+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736794+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:24:736795+0000","crush_location":[]}]}}' 2026-02-17T04:49:45.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:49:45.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:49:45.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:49:45.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:49:45.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:49:45.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:129: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:49:48.990 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-17T04:49:49.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:130: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:49:49.086 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-17T04:49:49.090 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:131: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:49:49.174 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-17T04:49:49.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:132: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-17T04:49:49.261 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-17T04:49:49.263 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:134: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:49:52.386 INFO:tasks.workunit.client.0.trial105.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T04:49:52.394 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:136: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:49:52.508 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:49:52.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:137: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:49:52.623 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:49:52.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:138: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:49:52.748 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.c 2026-02-17T04:49:52.756 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:139: TEST_4_mons_checks: ceph mon ok-to-rm d 2026-02-17T04:49:52.872 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.d 2026-02-17T04:49:52.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:49:52.881 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:52.881 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:52.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:52.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:52.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:52.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:52.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:49:52.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:49:52.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:49:52.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:49:52.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:52.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:52.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:52.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:52.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:52.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:52.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:49:52.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:49:52.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:49:52.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:49:52.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:49:52.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:49:52.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:52.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:49:52.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:49:53.000 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:53.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:49:53.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:49:53.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:49:53.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:49:53.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:49:53.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:49:53.015 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:49:53.015 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:49:53.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:49:53.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:49:53.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:49:53.016 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:49:53.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:49:53.017 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:49:53.017 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:49:53.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:49:53.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:49:53.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:49:53.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:53.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:49:53.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:49:53.021 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:49:53.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:49:53.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:49:53.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:49:53.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:49:53.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:49:53.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:49:53.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:49:53.026 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T04:49:53.027 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:49:53.027 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.027 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.027 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35170 2026-02-17T04:49:53.028 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:49:53.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:49:53.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:49:53.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T04:49:53.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_5_mons_checks td/ok-to-stop 2026-02-17T04:49:53.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:143: TEST_5_mons_checks: local dir=td/ok-to-stop 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:145: TEST_5_mons_checks: CEPH_ARGS='--fsid=5661606d-c59f-4576-bd11-863c6ffd0719 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153,127.0.0.1:7154 ' 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:147: TEST_5_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T04:49:53.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.051 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:53.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T04:49:53.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:53.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:49:53.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:49:53.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:53.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.082 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:49:53.083 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.083 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get fsid 2026-02-17T04:49:53.137 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:49:53.137 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.137 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:49:53.137 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:53.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:53.138 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:49:53.138 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.a.asok 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.139 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.a.asok config get mon_host 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:148: TEST_5_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T04:49:53.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T04:49:53.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:53.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:53.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:53.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:53.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.214 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:53.214 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T04:49:53.241 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:49:53.242 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:49:53.243 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.243 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.243 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.243 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:49:53.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get fsid 2026-02-17T04:49:53.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T04:49:53.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.299 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T04:49:53.299 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:53.299 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.b.asok 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.b.asok config get mon_host 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:149: TEST_5_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-17T04:49:53.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:53.373 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-17T04:49:53.397 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:53.398 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-17T04:49:53.398 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.398 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T04:49:53.398 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:53.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:53.400 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T04:49:53.400 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T04:49:53.400 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T04:49:53.400 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.401 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.401 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.401 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.c.asok 2026-02-17T04:49:53.401 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.401 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.c.asok config get fsid 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T04:49:53.455 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T04:49:53.456 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T04:49:53.456 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.457 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.457 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.457 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.c.asok 2026-02-17T04:49:53.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.457 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.c.asok config get mon_host 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:150: TEST_5_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-17T04:49:53.513 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-17T04:49:53.533 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:53.533 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:53.533 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:53.534 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:53.534 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.534 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.534 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:53.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-17T04:49:53.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:53.570 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-17T04:49:53.571 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.571 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T04:49:53.571 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:53.571 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:53.572 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T04:49:53.572 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T04:49:53.572 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T04:49:53.572 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.572 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.573 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.573 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.d.asok 2026-02-17T04:49:53.573 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.573 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.d.asok config get fsid 2026-02-17T04:49:53.627 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T04:49:53.628 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T04:49:53.629 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.629 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.630 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.630 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.d.asok 2026-02-17T04:49:53.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.d.asok config get mon_host 2026-02-17T04:49:53.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:151: TEST_5_mons_checks: run_mon td/ok-to-stop e --public-addr=127.0.0.1:7154 2026-02-17T04:49:53.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T04:49:53.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:49:53.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=e 2026-02-17T04:49:53.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:49:53.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/e 2026-02-17T04:49:53.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id e --mkfs --mon-data=td/ok-to-stop/e --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7154 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.703 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35170/$cluster-$name.asok' 2026-02-17T04:49:53.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id e --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/e '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.35170/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7154 2026-02-17T04:49:53.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:49:53.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e fsid 2026-02-17T04:49:53.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-17T04:49:53.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:49:53.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.e.asok 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.e.asok config get fsid 2026-02-17T04:49:53.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e mon_host 2026-02-17T04:49:53.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:49:53.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-17T04:49:53.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:49:53.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:49:53.793 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-17T04:49:53.793 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-17T04:49:53.793 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-17T04:49:53.794 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:49:53.794 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:49:53.794 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:49:53.794 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35170/ceph-mon.e.asok 2026-02-17T04:49:53.794 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:49:53.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35170/ceph-mon.e.asok config get mon_host 2026-02-17T04:49:53.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:152: TEST_5_mons_checks: wait_for_quorum 60 5 2026-02-17T04:49:53.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:49:53.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=5 2026-02-17T04:49:53.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:49:53.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 5 ]] 2026-02-17T04:49:53.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:49:53.850 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:49:53.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303853 2026-02-17T04:49:53.851 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:49:53.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303793 -lt 1771303853 ]] 2026-02-17T04:49:53.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-17T04:49:53.852 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:49:55.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:49:55.376 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:49:55.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-17T04:49:55.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:49:55.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:49:55.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:49:55.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:49:55.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:49:55.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:49:55.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:154: TEST_5_mons_checks: ceph mon ok-to-stop dne 2026-02-17T04:49:55.491 INFO:tasks.workunit.client.0.trial105.stderr:invalid mon(s) specified: [dne] 2026-02-17T04:49:55.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:155: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:49:55.615 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-17T04:49:55.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:156: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:49:58.759 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,c,d,e) after stopping [b] 2026-02-17T04:49:58.767 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:157: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:49:58.878 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b,d,e) after stopping [c] 2026-02-17T04:49:58.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:158: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-17T04:49:59.002 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b,c,e) after stopping [d] 2026-02-17T04:49:59.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:159: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-17T04:49:59.125 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b,c,d) after stopping [e] 2026-02-17T04:49:59.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:160: TEST_5_mons_checks: ceph mon ok-to-stop a b 2026-02-17T04:49:59.250 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (c,d,e) after stopping [a,b] 2026-02-17T04:49:59.260 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:161: TEST_5_mons_checks: ceph mon ok-to-stop c d 2026-02-17T04:49:59.364 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (a,b,e) after stopping [c,d] 2026-02-17T04:49:59.372 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:162: TEST_5_mons_checks: ceph mon ok-to-stop a b c 2026-02-17T04:49:59.472 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (d,e) after stopping mons [a,b,c] 2026-02-17T04:49:59.474 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:164: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:49:59.579 INFO:tasks.workunit.client.0.trial105.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T04:49:59.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:166: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:49:59.704 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:49:59.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:167: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:49:59.829 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:49:59.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:168: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:49:59.941 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.c 2026-02-17T04:49:59.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:169: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-17T04:50:00.068 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.d 2026-02-17T04:50:00.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:170: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-17T04:50:00.182 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.e 2026-02-17T04:50:00.192 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:172: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-17T04:50:00.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:00.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:00.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:00.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:00.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:00.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:173: TEST_5_mons_checks: wait_for_quorum 60 4 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:50:00.302 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:50:00.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303860 2026-02-17T04:50:00.303 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:00.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303800 -lt 1771303860 ]] 2026-02-17T04:50:00.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:00.305 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:00.416 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:00.416 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.417 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:00.417 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.428 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:00.428 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:00.429 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:00.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303800 -lt 1771303860 ]] 2026-02-17T04:50:00.430 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:00.430 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:00.545 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:00.545 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.545 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.545 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:00.557 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:00.557 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:00.557 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:00.558 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303800 -lt 1771303860 ]] 2026-02-17T04:50:00.558 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:00.559 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:00.667 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:00.667 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.668 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:00.668 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.677 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:00.677 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:00.678 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:00.679 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303800 -lt 1771303860 ]] 2026-02-17T04:50:00.679 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:00.679 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:00.791 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:00.791 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:00.792 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.801 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:00.801 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:00.801 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:00.802 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303800 -lt 1771303860 ]] 2026-02-17T04:50:00.802 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:00.803 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:00.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:00.914 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.914 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:00.914 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:00.924 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:00.924 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:00.925 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:00.926 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303800 -lt 1771303860 ]] 2026-02-17T04:50:00.926 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:00.926 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:01.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:01.041 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:01.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:01.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:01.053 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:01.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303801 -lt 1771303860 ]] 2026-02-17T04:50:01.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:01.054 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:01.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:01.167 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:01.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:01.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:01.177 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:01.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303801 -lt 1771303860 ]] 2026-02-17T04:50:01.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:01.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:01.295 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:01.296 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.296 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.296 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:01.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:01.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:01.305 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:01.307 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303801 -lt 1771303860 ]] 2026-02-17T04:50:01.307 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:01.307 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:01.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:01.419 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.419 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:01.420 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:01.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:01.429 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:01.430 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303801 -lt 1771303860 ]] 2026-02-17T04:50:01.430 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:01.431 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:01.544 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:01.544 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.545 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:01.545 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:01.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:01.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:01.554 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:01.555 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303801 -lt 1771303860 ]] 2026-02-17T04:50:01.555 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:01.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:04.685 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:04.685 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:04.686 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:04.687 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:04.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:04.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:04.697 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:04.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303804 -lt 1771303860 ]] 2026-02-17T04:50:04.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:04.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:04.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:04.812 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:04.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:04.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:04.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:04.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:04.822 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:04.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303804 -lt 1771303860 ]] 2026-02-17T04:50:04.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:04.823 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:04.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:04.934 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:04.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:04.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:04.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:04.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:04.945 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:04.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303804 -lt 1771303860 ]] 2026-02-17T04:50:04.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:04.946 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:05.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:05.059 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.060 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:05.060 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.069 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:05.069 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:05.070 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:05.070 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303805 -lt 1771303860 ]] 2026-02-17T04:50:05.071 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:05.071 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:05.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:05.187 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:05.197 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:05.197 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:05.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:05.198 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303805 -lt 1771303860 ]] 2026-02-17T04:50:05.198 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:05.199 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:05.314 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:05.314 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.315 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.315 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:05.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:05.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:05.325 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:05.326 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303805 -lt 1771303860 ]] 2026-02-17T04:50:05.326 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:05.326 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:05.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:05.438 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.438 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:05.439 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:05.448 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:05.448 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:05.449 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:05.449 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303805 -lt 1771303860 ]] 2026-02-17T04:50:05.450 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:05.450 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:08.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:08.564 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:08.565 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:08.565 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:08.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:08.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:08.573 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:08.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303808 -lt 1771303860 ]] 2026-02-17T04:50:08.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:08.575 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:08.685 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:08.685 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:08.686 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:08.686 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:08.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:08.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:08.696 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:08.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303808 -lt 1771303860 ]] 2026-02-17T04:50:08.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T04:50:08.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:15.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:15.443 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:15.443 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T04:50:15.444 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:15.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:50:15.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:50:15.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:50:15.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:50:15.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:50:15.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:175: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:50:15.557 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-17T04:50:15.565 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:176: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:50:15.674 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (c,d,e) after stopping [b] 2026-02-17T04:50:15.684 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:177: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:50:15.804 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,d,e) after stopping [c] 2026-02-17T04:50:15.814 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:178: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-17T04:50:15.920 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,e) after stopping [d] 2026-02-17T04:50:15.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:179: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-17T04:50:16.047 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-17T04:50:16.058 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:181: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:50:19.166 INFO:tasks.workunit.client.0.trial105.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T04:50:19.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:183: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:50:19.290 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:50:19.298 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:184: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:50:19.417 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.b 2026-02-17T04:50:19.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:185: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:50:19.543 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.c 2026-02-17T04:50:19.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:186: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-17T04:50:22.690 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.d 2026-02-17T04:50:22.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:187: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-17T04:50:22.812 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.e 2026-02-17T04:50:22.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:189: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.e 2026-02-17T04:50:22.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:22.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:22.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:22.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:22.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:22.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:22.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:190: TEST_5_mons_checks: wait_for_quorum 60 3 2026-02-17T04:50:22.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T04:50:22.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-17T04:50:22.930 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T04:50:22.931 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-17T04:50:22.931 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T04:50:22.931 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T04:50:22.931 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771303882 2026-02-17T04:50:22.931 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:22.932 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303822 -lt 1771303882 ]] 2026-02-17T04:50:22.932 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:22.932 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:23.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:23.048 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.048 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.049 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:23.058 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:23.058 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:23.058 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:23.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303823 -lt 1771303882 ]] 2026-02-17T04:50:23.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:23.060 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:23.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:23.171 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:23.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:23.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:23.181 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:23.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303823 -lt 1771303882 ]] 2026-02-17T04:50:23.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:23.183 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:23.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:23.300 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.301 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:23.301 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.310 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:23.310 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:23.311 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:23.311 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303823 -lt 1771303882 ]] 2026-02-17T04:50:23.311 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:23.312 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:23.422 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:23.423 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.423 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:23.423 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.433 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:23.433 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:23.433 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:23.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303823 -lt 1771303882 ]] 2026-02-17T04:50:23.435 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:23.435 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:23.547 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:23.548 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.548 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:23.548 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:23.557 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:23.557 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:23.558 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:23.559 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303823 -lt 1771303882 ]] 2026-02-17T04:50:23.559 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:23.559 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:26.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:26.672 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:26.673 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:26.673 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:26.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:26.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:26.683 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:26.684 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303826 -lt 1771303882 ]] 2026-02-17T04:50:26.684 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:26.684 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:29.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:29.797 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:29.798 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:29.798 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:29.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T04:50:29.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T04:50:29.808 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T04:50:29.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771303829 -lt 1771303882 ]] 2026-02-17T04:50:29.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T04:50:29.810 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T04:50:38.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T04:50:38.944 INFO:tasks.workunit.client.0.trial105.stderr:{"election_epoch":10,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:38.945 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T04:50:38.945 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"5661606d-c59f-4576-bd11-863c6ffd0719","modified":"2026-02-17T04:49:53.042812Z","created":"2026-02-17T04:49:53.042812Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042805+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042809+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042810+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T04:49:53:042811+0000","crush_location":[]}]}}' 2026-02-17T04:50:38.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T04:50:38.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T04:50:38.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T04:50:38.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T04:50:38.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T04:50:38.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:192: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-17T04:50:39.062 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-17T04:50:39.072 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:193: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-17T04:50:39.160 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-17T04:50:39.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:194: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-17T04:50:39.249 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-17T04:50:39.252 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:195: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-17T04:50:39.337 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-17T04:50:39.340 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:196: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-17T04:50:39.443 INFO:tasks.workunit.client.0.trial105.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-17T04:50:39.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:198: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-17T04:50:39.550 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-17T04:50:39.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:200: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-17T04:50:39.655 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.a 2026-02-17T04:50:39.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:201: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-17T04:50:42.785 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: removing mon.b would break quorum 2026-02-17T04:50:42.787 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:202: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-17T04:50:45.876 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-17T04:50:45.879 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:203: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-17T04:50:45.966 INFO:tasks.workunit.client.0.trial105.stderr:Error EBUSY: removing mon.d would break quorum 2026-02-17T04:50:45.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:204: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-17T04:50:46.072 INFO:tasks.workunit.client.0.trial105.stderr:safe to remove mon.e 2026-02-17T04:50:46.081 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T04:50:46.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:46.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:46.081 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:46.081 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:46.081 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:46.190 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:46.190 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:46.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:46.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:46.196 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:50:46.197 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:50:46.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:50:46.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:50:46.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:50:46.199 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:50:46.200 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:46.201 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:50:46.201 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:50:46.201 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:46.202 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:50:46.203 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:50:46.203 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:50:46.218 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:50:46.218 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.218 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:50:46.218 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:50:46.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:50:46.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:50:46.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T04:50:46.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/ok-to-stop 0 2026-02-17T04:50:46.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:46.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:46.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:46.222 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:50:46.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:50:46.224 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:50:46.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:50:46.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:50:46.226 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:50:46.226 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:46.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:50:46.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:50:46.227 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:46.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:50:46.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T04:50:46.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T04:50:46.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:50:46.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35170 2026-02-17T04:50:46.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35170 2026-02-17T04:50:46.232 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:50:46.232 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:50:46.232 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T04:50:46.234 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T04:50:46.234 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T04:50:46.285 INFO:tasks.workunit:Running workunit misc/rados-striper.sh... 2026-02-17T04:50:46.285 DEBUG:teuthology.orchestra.run.trial105:workunit test misc/rados-striper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh 2026-02-17T04:50:46.338 INFO:tasks.workunit.client.0.trial105.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T04:50:46.341 INFO:tasks.workunit.client.0.trial105.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T04:50:46.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/rados-striper 2026-02-17T04:50:46.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:20: run: local dir=td/rados-striper 2026-02-17T04:50:46.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:21: run: shift 2026-02-17T04:50:46.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: export CEPH_MON=127.0.0.1:7116 2026-02-17T04:50:46.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: CEPH_MON=127.0.0.1:7116 2026-02-17T04:50:46.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:24: run: export CEPH_ARGS 2026-02-17T04:50:46.343 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: uuidgen 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: CEPH_ARGS+='--fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none ' 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7116 ' 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:29: run: setup td/rados-striper 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/rados-striper 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/rados-striper 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:46.345 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:46.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:46.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:46.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:46.346 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:46.346 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:50:46.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:50:46.347 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:50:46.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:50:46.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:50:46.348 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:50:46.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:46.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:50:46.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:50:46.350 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:46.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:50:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:50:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-17T04:50:46.353 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:50:46.353 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.353 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:46.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.51139 2026-02-17T04:50:46.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:50:46.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:50:46.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/rados-striper 2026-02-17T04:50:46.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:50:46.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.356 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:46.356 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.51139 2026-02-17T04:50:46.357 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:50:46.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T04:50:46.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T04:50:46.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/rados-striper 1' TERM HUP INT 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:32: run: run_mon td/rados-striper a 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/rados-striper 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/rados-striper/a 2026-02-17T04:50:46.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/rados-striper/a --run-dir=td/rados-striper 2026-02-17T04:50:46.383 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:50:46.383 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:46.383 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:46.383 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:46.384 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.384 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:46.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:46.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/rados-striper/a '--log-file=td/rados-striper/$name.log' '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --mon-cluster-log-file=td/rados-striper/log --run-dir=td/rados-striper '--pid-file=td/rados-striper/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:50:46.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:50:46.415 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:50:46.415 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:50:46.415 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:50:46.415 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:50:46.415 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:50:46.416 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.51139/ceph-mon.a.asok 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:50:46.417 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.51139/ceph-mon.a.asok config get fsid 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:50:46.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:50:46.476 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:50:46.476 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.476 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:46.476 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.51139/ceph-mon.a.asok 2026-02-17T04:50:46.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:50:46.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.51139/ceph-mon.a.asok config get mon_host 2026-02-17T04:50:46.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:33: run: run_osd td/rados-striper 0 2026-02-17T04:50:46.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-17T04:50:46.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:50:46.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:50:46.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/0 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/0' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/0/journal' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-17T04:50:46.533 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:50:46.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:50:46.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/0 2026-02-17T04:50:46.536 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:50:46.536 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7d7cceff-7746-48e3-ba72-da41607568e9 2026-02-17T04:50:46.536 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7d7cceff-7746-48e3-ba72-da41607568e9' 2026-02-17T04:50:46.537 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 7d7cceff-7746-48e3-ba72-da41607568e9 2026-02-17T04:50:46.537 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:50:46.547 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCm85NpQe1+IBAAeYSeciIPByzgCg7ccOAG9g== 2026-02-17T04:50:46.547 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCm85NpQe1+IBAAeYSeciIPByzgCg7ccOAG9g=="}' 2026-02-17T04:50:46.547 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7d7cceff-7746-48e3-ba72-da41607568e9 -i td/rados-striper/0/new.json 2026-02-17T04:50:46.667 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:50:46.674 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/0/new.json 2026-02-17T04:50:46.675 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCm85NpQe1+IBAAeYSeciIPByzgCg7ccOAG9g== --osd-uuid 7d7cceff-7746-48e3-ba72-da41607568e9 2026-02-17T04:50:46.691 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:46.686+0000 7feda7d30a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:46.693 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:46.690+0000 7feda7d30a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:46.694 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:46.690+0000 7feda7d30a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:46.694 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:46.690+0000 7feda7d30a40 -1 bdev(0x56177cca6400 td/rados-striper/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:50:46.695 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:46.690+0000 7feda7d30a40 -1 bluestore(td/rados-striper/0) _read_fsid unparsable uuid 2026-02-17T04:50:47.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/0/keyring 2026-02-17T04:50:47.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:50:47.290 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:50:47.290 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:50:47.290 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:50:47.411 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:50:47.412 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:50:47.412 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:50:47.413 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:50:47.413 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:50:47.413 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:50:47.433 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:47.430+0000 7fceaa77aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:47.435 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:47.430+0000 7fceaa77aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:47.436 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:47.430+0000 7fceaa77aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:50:47.520 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:50:47.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:47.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:50:47.628 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:50:47.794 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:47.790+0000 7fceaa77aa40 -1 Falling back to public interface 2026-02-17T04:50:48.323 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:48.318+0000 7fceaa77aa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:50:48.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:50:48.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:48.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:50:48.631 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:50:48.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:48.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:50:48.741 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:50:49.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:50:49.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:49.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:50:49.743 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:50:49.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:49.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:50:49.853 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1500027990,v1:127.0.0.1:6801/1500027990] [v2:127.0.0.1:6802/1500027990,v1:127.0.0.1:6803/1500027990] exists,up 7d7cceff-7746-48e3-ba72-da41607568e9 2026-02-17T04:50:49.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:50:49.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:50:49.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:34: run: run_osd td/rados-striper 1 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/1 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/1' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/1/journal' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:49.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:50:49.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/1 2026-02-17T04:50:49.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:50:49.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=37f8320c-ba3b-478a-b7ba-7d6fa371000a 2026-02-17T04:50:49.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 37f8320c-ba3b-478a-b7ba-7d6fa371000a' 2026-02-17T04:50:49.857 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 37f8320c-ba3b-478a-b7ba-7d6fa371000a 2026-02-17T04:50:49.857 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:50:49.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCp85NpMnbAMxAAydbqLZALnQVus6zQlC8rlg== 2026-02-17T04:50:49.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCp85NpMnbAMxAAydbqLZALnQVus6zQlC8rlg=="}' 2026-02-17T04:50:49.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 37f8320c-ba3b-478a-b7ba-7d6fa371000a -i td/rados-striper/1/new.json 2026-02-17T04:50:49.982 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:50:49.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/1/new.json 2026-02-17T04:50:49.991 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCp85NpMnbAMxAAydbqLZALnQVus6zQlC8rlg== --osd-uuid 37f8320c-ba3b-478a-b7ba-7d6fa371000a 2026-02-17T04:50:50.006 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.002+0000 7f591296da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:50.008 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.002+0000 7f591296da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:50.009 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.006+0000 7f591296da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:50.009 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.006+0000 7f591296da40 -1 bdev(0x562123fd2400 td/rados-striper/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:50:50.009 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.006+0000 7f591296da40 -1 bluestore(td/rados-striper/1) _read_fsid unparsable uuid 2026-02-17T04:50:50.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/1/keyring 2026-02-17T04:50:50.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:50:50.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:50:50.665 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:50:50.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:50:50.779 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:50:50.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:50:50.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:50:50.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:50:50.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:50:50.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:50:50.795 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.790+0000 7f334617da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:50.796 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.790+0000 7f334617da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:50.797 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:50.794+0000 7f334617da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:50.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:50:50.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:50:50.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:50:50.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:50:50.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:50:50.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:50.887 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:50:50.888 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:50:50.888 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:50.888 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:50:50.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:50:51.131 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:51.126+0000 7f334617da40 -1 Falling back to public interface 2026-02-17T04:50:51.734 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:51.730+0000 7f334617da40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:50:52.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:50:52.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:52.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:50:52.001 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:50:52.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:52.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:50:52.106 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:50:53.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:50:53.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:53.108 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:50:53.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:50:53.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:53.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:50:53.217 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/3846854059,v1:127.0.0.1:6809/3846854059] [v2:127.0.0.1:6810/3846854059,v1:127.0.0.1:6811/3846854059] exists,up 37f8320c-ba3b-478a-b7ba-7d6fa371000a 2026-02-17T04:50:53.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:50:53.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:35: run: run_osd td/rados-striper 2 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/2 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:50:53.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/2' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/2/journal' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:53.222 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:50:53.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/2 2026-02-17T04:50:53.224 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:50:53.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2891d130-7547-4685-996b-407110d1b591 2026-02-17T04:50:53.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2891d130-7547-4685-996b-407110d1b591' 2026-02-17T04:50:53.225 INFO:tasks.workunit.client.0.trial105.stdout:add osd2 2891d130-7547-4685-996b-407110d1b591 2026-02-17T04:50:53.225 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:50:53.234 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCt85Np7nTcDRAA0wxba2qmL21/o8TiTJbUlA== 2026-02-17T04:50:53.234 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCt85Np7nTcDRAA0wxba2qmL21/o8TiTJbUlA=="}' 2026-02-17T04:50:53.234 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2891d130-7547-4685-996b-407110d1b591 -i td/rados-striper/2/new.json 2026-02-17T04:50:53.350 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:50:53.360 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/2/new.json 2026-02-17T04:50:53.361 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCt85Np7nTcDRAA0wxba2qmL21/o8TiTJbUlA== --osd-uuid 2891d130-7547-4685-996b-407110d1b591 2026-02-17T04:50:53.384 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:53.378+0000 7f9ab01e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:53.386 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:53.382+0000 7f9ab01e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:53.387 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:53.382+0000 7f9ab01e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:53.387 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:53.382+0000 7f9ab01e1a40 -1 bdev(0x55cb73840400 td/rados-striper/2/block) open stat got: (1) Operation not permitted 2026-02-17T04:50:53.388 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:53.382+0000 7f9ab01e1a40 -1 bluestore(td/rados-striper/2) _read_fsid unparsable uuid 2026-02-17T04:50:54.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/2/keyring 2026-02-17T04:50:54.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:50:54.053 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T04:50:54.054 INFO:tasks.workunit.client.0.trial105.stdout:adding osd2 key to auth repository 2026-02-17T04:50:54.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:50:54.174 INFO:tasks.workunit.client.0.trial105.stdout:start osd.2 2026-02-17T04:50:54.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T04:50:54.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=a95807f2-bab9-49c7-90ae-589789d85831 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.51139/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:50:54.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:50:54.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:50:54.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:50:54.195 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:54.190+0000 7fc592630a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:54.197 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:54.194+0000 7fc592630a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:54.198 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:54.194+0000 7fc592630a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T04:50:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:50:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T04:50:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:50:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:50:54.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:54.282 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:50:54.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:50:54.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:54.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:50:54.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:50:54.538 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:54.534+0000 7fc592630a40 -1 Falling back to public interface 2026-02-17T04:50:55.122 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:55.118+0000 7fc592630a40 -1 osd.2 0 log_to_monitors true 2026-02-17T04:50:55.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:50:55.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:55.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:50:55.390 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:50:55.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:55.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:50:55.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:50:56.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:50:56.501 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:50:56.501 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:50:56.501 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:50:56.501 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:50:56.501 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T04:50:56.614 INFO:tasks.workunit.client.0.trial105.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/68644417,v1:127.0.0.1:6817/68644417] [v2:127.0.0.1:6818/68644417,v1:127.0.0.1:6819/68644417] exists,up 2891d130-7547-4685-996b-407110d1b591 2026-02-17T04:50:56.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:50:56.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:50:56.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:50:56.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:36: run: create_rbd_pool 2026-02-17T04:50:56.614 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:50:56.719 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:50:56.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:50:56.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:50:56.858 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:50:56.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:50:57.869 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:50:58.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:39: run: dd if=/dev/urandom of=td/rados-striper/toyfile bs=1234 count=1 2026-02-17T04:50:58.150 INFO:tasks.workunit.client.0.trial105.stderr:1+0 records in 2026-02-17T04:50:58.150 INFO:tasks.workunit.client.0.trial105.stderr:1+0 records out 2026-02-17T04:50:58.150 INFO:tasks.workunit.client.0.trial105.stderr:1234 bytes (1.2 kB, 1.2 KiB) copied, 3.9473e-05 s, 31.3 MB/s 2026-02-17T04:50:58.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:42: run: rados --pool rbd --striper put toyfile td/rados-striper/toyfile 2026-02-17T04:50:58.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: rados --pool rbd --striper stat toyfile 2026-02-17T04:50:58.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: cut -d , -f 2 2026-02-17T04:50:58.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-17T04:50:58.196 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: cut -d , -f 2 2026-02-17T04:50:58.218 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:47: run: echo ' size 1234' 2026-02-17T04:50:58.218 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:48: run: diff -w td/rados-striper/stripedStat td/rados-striper/refstat 2026-02-17T04:50:58.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:49: run: diff -w td/rados-striper/stat td/rados-striper/refstat 2026-02-17T04:50:58.221 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:50: run: rados --pool rbd stat toyfile 2026-02-17T04:50:58.244 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:51: run: grep -q 'No such file or directory' td/rados-striper/staterror 2026-02-17T04:50:58.245 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:54: run: rados --pool rbd --striper get toyfile td/rados-striper/stripedGroup 2026-02-17T04:50:58.264 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:55: run: diff -w td/rados-striper/toyfile td/rados-striper/stripedGroup 2026-02-17T04:50:58.264 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:56: run: rados --pool rbd get toyfile.0000000000000000 td/rados-striper/nonSTripedGroup 2026-02-17T04:50:58.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:57: run: diff -w td/rados-striper/toyfile td/rados-striper/nonSTripedGroup 2026-02-17T04:50:58.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:60: run: rados --pool rbd --striper truncate toyfile 12 2026-02-17T04:50:58.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: rados --pool rbd --striper stat toyfile 2026-02-17T04:50:58.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: cut -d , -f 2 2026-02-17T04:50:58.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-17T04:50:58.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: cut -d , -f 2 2026-02-17T04:50:58.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:63: run: echo ' size 12' 2026-02-17T04:50:58.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:64: run: diff -w td/rados-striper/stripedStat td/rados-striper/reftrunc 2026-02-17T04:50:58.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:65: run: diff -w td/rados-striper/stat td/rados-striper/reftrunc 2026-02-17T04:50:58.348 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:69: run: rados --pool rbd --striper setxattr toyfile somexattr somevalue 2026-02-17T04:50:58.371 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:70: run: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-17T04:50:58.395 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:71: run: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-17T04:50:58.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:72: run: echo somevalue 2026-02-17T04:50:58.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:73: run: diff -w td/rados-striper/xattrvalue td/rados-striper/refvalue 2026-02-17T04:50:58.420 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:74: run: diff -w td/rados-striper/xattrvalue2 td/rados-striper/refvalue 2026-02-17T04:50:58.421 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:75: run: rados --pool rbd --striper listxattr toyfile 2026-02-17T04:50:58.444 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:76: run: echo somexattr 2026-02-17T04:50:58.444 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:77: run: diff -w td/rados-striper/xattrlist td/rados-striper/reflist 2026-02-17T04:50:58.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: rados --pool rbd listxattr toyfile.0000000000000000 2026-02-17T04:50:58.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: grep -v striper 2026-02-17T04:50:58.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:79: run: diff -w td/rados-striper/xattrlist2 td/rados-striper/reflist 2026-02-17T04:50:58.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:80: run: rados --pool rbd --striper rmxattr toyfile somexattr 2026-02-17T04:50:58.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:82: run: local 'attr_not_found_str=No data available' 2026-02-17T04:50:58.490 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: uname 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: '[' Linux = FreeBSD ']' 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:85: run: expect_failure td/rados-striper 'No data available' rados --pool rbd --striper getxattr toyfile somexattr 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:50:58.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:50:58.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-17T04:50:58.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T04:50:58.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T04:50:58.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:87: run: expect_failure td/rados-striper 'No data available' rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:50:58.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-17T04:50:58.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T04:50:58.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T04:50:58.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-17T04:50:58.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T04:50:58.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:91: run: rados --pool rbd --striper rm toyfile 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:92: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd --striper stat toyfile 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:50:58.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper stat toyfile 2026-02-17T04:50:58.577 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T04:50:58.577 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T04:50:58.577 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-17T04:50:58.578 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T04:50:58.578 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:94: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd stat toyfile.0000000000000000 2026-02-17T04:50:58.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T04:50:58.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:50:58.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-17T04:50:58.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:50:58.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:50:58.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd stat toyfile.0000000000000000 2026-02-17T04:50:58.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T04:50:58.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T04:50:58.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-17T04:50:58.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:98: run: teardown td/rados-striper 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:58.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:58.710 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:58.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:50:58.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:50:58.712 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:50:58.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:50:58.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:50:58.713 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:50:58.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:58.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:50:58.715 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:50:58.715 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:58.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:50:58.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:50:58.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-17T04:50:58.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:50:58.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:58.726 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.51139 2026-02-17T04:50:58.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:50:58.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/rados-striper 0 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:58.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:58.730 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:58.731 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:50:58.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:50:58.732 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:50:58.732 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:50:58.732 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:50:58.733 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:50:58.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:58.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:50:58.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:50:58.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:58.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:50:58.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T04:50:58.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-17T04:50:58.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:50:58.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.51139 2026-02-17T04:50:58.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.51139 2026-02-17T04:50:58.740 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:50:58.740 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:50:58.740 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T04:50:58.740 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T04:50:58.740 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T04:50:58.789 INFO:tasks.workunit:Running workunit misc/test-ceph-helpers.sh... 2026-02-17T04:50:58.789 DEBUG:teuthology.orchestra.run.trial105:workunit test misc/test-ceph-helpers.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh 2026-02-17T04:50:58.844 INFO:tasks.workunit.client.0.trial105.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T04:50:58.847 INFO:tasks.workunit.client.0.trial105.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T04:50:58.847 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: export PATH=./bin:.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:50:58.847 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: PATH=./bin:.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T04:50:58.847 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: export CEPH_MON=127.0.0.1:7109 2026-02-17T04:50:58.848 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: CEPH_MON=127.0.0.1:7109 2026-02-17T04:50:58.848 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: export CEPH_ARGS 2026-02-17T04:50:58.848 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: uuidgen 2026-02-17T04:50:58.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: CEPH_ARGS+=' --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none ' 2026-02-17T04:50:58.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: CEPH_ARGS+='--mon-host=127.0.0.1:7109 ' 2026-02-17T04:50:58.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: export CEPH_CONF=/dev/null 2026-02-17T04:50:58.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: CEPH_CONF=/dev/null 2026-02-17T04:50:58.849 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: set 2026-02-17T04:50:58.849 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: sed -n -e 's/^\(test_[0-9a-z_]*\) .*/\1/p' 2026-02-17T04:50:58.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: local 'funcs=test_activate_osd 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_activate_osd_after_mark_down 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_activate_osd_skip_benchmark 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_destroy_osd 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_display_logs 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_erasure_code_plugin_exists 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_expect_failure 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_file_with_random_data 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_flush_pg_stats 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_get_config 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_get_is_making_recovery_progress 2026-02-17T04:50:58.852 INFO:tasks.workunit.client.0.trial105.stderr:test_get_last_scrub_stamp 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_not_primary 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_num_active_clean 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_num_active_or_peered 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_num_pgs 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_op_scheduler 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_osd_id_used_by_pgs 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_osds 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_pg 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_primary 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_get_timeout_delays 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_is_clean 2026-02-17T04:50:58.853 INFO:tasks.workunit.client.0.trial105.stderr:test_kill_daemon 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_kill_daemons 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_objectstore_tool 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_pg_schedule_scrub 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_pg_scrub 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_repair 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_run_in_background 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_run_mon 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_run_osd 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_set_config 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_setup 2026-02-17T04:50:58.854 INFO:tasks.workunit.client.0.trial105.stderr:test_teardown 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_background 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_for_clean 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_for_health_ok 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_for_osd 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_for_peered 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_for_scrub 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:test_wait_osd_id_used_by_pgs' 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: local dir=td/ceph-helpers 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd td/ceph-helpers 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: test_activate_osd: local dir=td/ceph-helpers 2026-02-17T04:50:58.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: test_activate_osd: setup td/ceph-helpers 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:50:58.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:50:58.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:58.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:50:58.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:50:58.859 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:50:58.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:50:58.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:50:58.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:50:58.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:50:58.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:58.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:50:58.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:50:58.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:50:58.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:50:58.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:50:58.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:58.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:50:58.865 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:50:58.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T04:50:58.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T04:50:58.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:881: test_activate_osd: run_mon td/ceph-helpers a 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:50:58.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:50:58.892 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:50:58.921 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:50:58.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.923 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:58.924 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:50:58.924 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:50:58.924 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:50:58.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:50:58.980 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:50:58.981 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:50:58.981 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:58.981 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:58.981 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:50:58.981 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:50:58.982 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:50:59.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: test_activate_osd: run_mgr td/ceph-helpers x 2026-02-17T04:50:59.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:50:59.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:50:59.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:50:59.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:50:59.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:50:59.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:50:59.148 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:50:59.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:59.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:59.149 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:59.149 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:59.149 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:59.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:50:59.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:50:59.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:884: test_activate_osd: run_osd td/ceph-helpers 0 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:50:59.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:50:59.172 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:50:59.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:50:59.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:50:59.174 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:50:59.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c04ae473-c920-4385-b1e1-18ccdfc28ab2 2026-02-17T04:50:59.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c04ae473-c920-4385-b1e1-18ccdfc28ab2' 2026-02-17T04:50:59.175 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 c04ae473-c920-4385-b1e1-18ccdfc28ab2 2026-02-17T04:50:59.175 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:50:59.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCz85NpMbvHChAAxiWfU9GIsPXvrA6m5is5Kw== 2026-02-17T04:50:59.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCz85NpMbvHChAAxiWfU9GIsPXvrA6m5is5Kw=="}' 2026-02-17T04:50:59.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c04ae473-c920-4385-b1e1-18ccdfc28ab2 -i td/ceph-helpers/0/new.json 2026-02-17T04:50:59.292 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:50:59.300 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:50:59.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCz85NpMbvHChAAxiWfU9GIsPXvrA6m5is5Kw== --osd-uuid c04ae473-c920-4385-b1e1-18ccdfc28ab2 2026-02-17T04:50:59.317 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:59.314+0000 7ff2116c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:59.319 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:59.314+0000 7ff2116c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:59.320 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:59.314+0000 7ff2116c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:50:59.320 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:59.314+0000 7ff2116c8a40 -1 bdev(0x558e6b256400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:50:59.321 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:50:59.314+0000 7ff2116c8a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:00.104 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:00.105 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:00.105 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:00.106 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:00.106 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:00.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:00.231 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:00.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:00.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:00.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:00.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:00.247 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:00.242+0000 7fe982affa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:00.248 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:00.242+0000 7fe982affa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:00.249 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:00.246+0000 7fe982affa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:00.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:00.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:00.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:00.602 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:00.598+0000 7fe982affa40 -1 Falling back to public interface 2026-02-17T04:51:01.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:01.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:01.471 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:01.471 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:01.471 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:01.471 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:01.644 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:01.638+0000 7fe982affa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:01.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:02.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:02.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:02.698 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:02.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:02.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:02.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:02.915 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3517487274,v1:127.0.0.1:6801/3517487274] [v2:127.0.0.1:6802/3517487274,v1:127.0.0.1:6803/3517487274] exists,up c04ae473-c920-4385-b1e1-18ccdfc28ab2 2026-02-17T04:51:02.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:02.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:02.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:02.916 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: get_asok_path osd.0 2026-02-17T04:51:02.916 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:02.917 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:02.918 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:02.918 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:02.918 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:02.918 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:02.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: CEPH_ARGS= 2026-02-17T04:51:02.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:51:02.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-17T04:51:02.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: echo '{"osd_max_backfills":"1"}' 2026-02-17T04:51:02.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: grep --quiet osd_max_backfills 2026-02-17T04:51:02.978 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:889: test_activate_osd: kill_daemons td/ceph-helpers TERM osd 2026-02-17T04:51:02.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:02.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:02.979 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:02.979 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:02.979 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:03.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:03.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:891: test_activate_osd: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:03.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-17T04:51:03.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:03.289 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:03.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T04:51:03.289 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-17T04:51:03.289 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T04:51:03.290 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T04:51:03.290 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T04:51:03.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T04:51:03.291 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T04:51:03.309 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:03.306+0000 7f2c5d262a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:03.311 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:03.306+0000 7f2c5d262a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:03.312 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:03.306+0000 7f2c5d262a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:03.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:03.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:03.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:03.730 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:03.726+0000 7f2c5d262a40 -1 Falling back to public interface 2026-02-17T04:51:04.353 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:04.346+0000 7f2c5d262a40 -1 osd.0 5 log_to_monitors true 2026-02-17T04:51:04.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:04.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:04.726 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:04.727 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:04.727 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:04.727 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:04.939 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1868926747,v1:127.0.0.1:6801/1868926747] [v2:127.0.0.1:6802/1868926747,v1:127.0.0.1:6803/1868926747] exists,up c04ae473-c920-4385-b1e1-18ccdfc28ab2 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: get_asok_path osd.0 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:04.940 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:04.941 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:04.941 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:04.941 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:04.941 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: CEPH_ARGS= 2026-02-17T04:51:04.942 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:51:05.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:894: test_activate_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:896: test_activate_osd: teardown td/ceph-helpers 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:05.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:05.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:05.108 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:05.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:05.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:05.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:05.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:05.111 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:05.112 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:05.112 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:05.113 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:05.113 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:05.113 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:05.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:05.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:05.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:05.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.118 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:05.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:05.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:05.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:05.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_after_mark_down td/ceph-helpers 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:900: test_activate_osd_after_mark_down: local dir=td/ceph-helpers 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:902: test_activate_osd_after_mark_down: setup td/ceph-helpers 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:05.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:05.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:05.123 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:05.123 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:05.124 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:05.125 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:05.125 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:05.125 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:05.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:05.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:05.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:05.127 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:05.128 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:05.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:05.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:05.131 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:05.131 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.131 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.131 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:05.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:05.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:05.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:05.134 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:05.134 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.134 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.134 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:05.135 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:05.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:05.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:904: test_activate_osd_after_mark_down: run_mon td/ceph-helpers a 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:05.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:51:05.157 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:05.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:05.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:05.158 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:05.158 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.158 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:05.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:51:05.189 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:05.192 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:05.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:05.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:05.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:05.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:05.193 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:05.194 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:05.195 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:05.257 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.258 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.259 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:05.259 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:05.259 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:05.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:905: test_activate_osd_after_mark_down: run_mgr td/ceph-helpers x 2026-02-17T04:51:05.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:05.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:05.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:05.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:05.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:05.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:05.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:05.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:05.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:05.426 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:05.427 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.427 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.427 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:05.427 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:05.428 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:05.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:907: test_activate_osd_after_mark_down: run_osd td/ceph-helpers 0 2026-02-17T04:51:05.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:05.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:05.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:05.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:05.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=09e3e48c-1212-443c-ab09-d58066fe2370 2026-02-17T04:51:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 09e3e48c-1212-443c-ab09-d58066fe2370' 2026-02-17T04:51:05.455 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 09e3e48c-1212-443c-ab09-d58066fe2370 2026-02-17T04:51:05.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:05.463 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC585NpGEuAGxAA8Y5EJzQQ7d51zvFY6Tlvmw== 2026-02-17T04:51:05.463 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC585NpGEuAGxAA8Y5EJzQQ7d51zvFY6Tlvmw=="}' 2026-02-17T04:51:05.463 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 09e3e48c-1212-443c-ab09-d58066fe2370 -i td/ceph-helpers/0/new.json 2026-02-17T04:51:05.574 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:05.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:05.583 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC585NpGEuAGxAA8Y5EJzQQ7d51zvFY6Tlvmw== --osd-uuid 09e3e48c-1212-443c-ab09-d58066fe2370 2026-02-17T04:51:05.599 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:05.594+0000 7f6e38ff4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:05.600 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:05.594+0000 7f6e38ff4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:05.601 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:05.598+0000 7f6e38ff4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:05.602 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:05.598+0000 7f6e38ff4a40 -1 bdev(0x559cedf24400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:05.602 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:05.598+0000 7f6e38ff4a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:06.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:06.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:06.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:06.318 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:06.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:06.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:06.443 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:06.443 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:06.443 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:06.444 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:06.444 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:06.467 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:06.462+0000 7f1f27ee0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:06.469 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:06.466+0000 7f1f27ee0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:06.470 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:06.466+0000 7f1f27ee0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:06.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:06.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:06.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:06.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:06.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:06.552 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:06.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:06.553 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:06.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:06.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:06.666 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:06.998 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:06.994+0000 7f1f27ee0a40 -1 Falling back to public interface 2026-02-17T04:51:07.667 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:07.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:07.668 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:07.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:07.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:07.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:07.776 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:07.974 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:07.970+0000 7f1f27ee0a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:08.778 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:08.778 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:08.778 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:08.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:08.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:08.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:08.993 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1638305631,v1:127.0.0.1:6801/1638305631] [v2:127.0.0.1:6802/1638305631,v1:127.0.0.1:6803/1638305631] exists,up 09e3e48c-1212-443c-ab09-d58066fe2370 2026-02-17T04:51:08.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:08.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:08.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:08.994 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-17T04:51:08.994 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:08.994 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:08.994 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:08.995 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:08.995 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:08.995 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:08.995 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-17T04:51:08.995 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:51:09.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"1"}' 2026-02-17T04:51:09.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: echo '{"osd_max_backfills":"1"}' 2026-02-17T04:51:09.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: grep --quiet osd_max_backfills 2026-02-17T04:51:09.056 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:912: test_activate_osd_after_mark_down: kill_daemons td/ceph-helpers TERM osd 2026-02-17T04:51:09.057 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:09.057 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:09.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:09.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:09.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:09.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:09.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:913: test_activate_osd_after_mark_down: ceph osd down 0 2026-02-17T04:51:09.576 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 is already down. 2026-02-17T04:51:09.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:914: test_activate_osd_after_mark_down: wait_for_osd down 0 2026-02-17T04:51:09.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-17T04:51:09.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:09.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:09.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:09.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:09.587 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:09.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:09.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:09.588 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-17T04:51:09.819 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 7 last_clean_interval [0,0) [v2:127.0.0.1:6800/1638305631,v1:127.0.0.1:6801/1638305631] [v2:127.0.0.1:6802/1638305631,v1:127.0.0.1:6803/1638305631] exists 09e3e48c-1212-443c-ab09-d58066fe2370 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: test_activate_osd_after_mark_down: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T04:51:09.823 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:09.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:09.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:09.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:09.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T04:51:09.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-17T04:51:09.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:09.826 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:09.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T04:51:09.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-17T04:51:09.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T04:51:09.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T04:51:09.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T04:51:09.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T04:51:09.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T04:51:09.841 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:09.838+0000 7f475791aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:09.843 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:09.838+0000 7f475791aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:09.844 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:09.838+0000 7f475791aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:10.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T04:51:10.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:10.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:10.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:10.037 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:10.037 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:10.037 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:10.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:10.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:10.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:10.248 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:10.302 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:10.298+0000 7f475791aa40 -1 Falling back to public interface 2026-02-17T04:51:10.971 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:10.966+0000 7f475791aa40 -1 osd.0 6 log_to_monitors true 2026-02-17T04:51:11.249 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:11.250 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:11.250 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:11.250 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:11.250 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:11.250 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:11.460 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 7 last_clean_interval [5,6) [v2:127.0.0.1:6800/1095857712,v1:127.0.0.1:6801/1095857712] [v2:127.0.0.1:6802/1095857712,v1:127.0.0.1:6803/1095857712] exists,up 09e3e48c-1212-443c-ab09-d58066fe2370 2026-02-17T04:51:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:11.462 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:11.462 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-17T04:51:11.462 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:11.462 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:11.463 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:11.463 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.463 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.463 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:11.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-17T04:51:11.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T04:51:11.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"20"}' 2026-02-17T04:51:11.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: test_activate_osd_after_mark_down: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-17T04:51:11.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: test_activate_osd_after_mark_down: teardown td/ceph-helpers 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:11.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:11.628 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:11.629 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:11.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:11.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:11.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:11.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:11.632 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:11.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:11.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:11.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:11.633 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:11.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:11.636 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:11.636 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:11.640 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:11.641 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.641 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:11.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:11.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:11.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:11.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_skip_benchmark td/ceph-helpers 2026-02-17T04:51:11.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: test_activate_osd_skip_benchmark: local dir=td/ceph-helpers 2026-02-17T04:51:11.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:927: test_activate_osd_skip_benchmark: setup td/ceph-helpers 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:11.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:11.645 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:11.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:11.646 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:11.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:11.647 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:11.647 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:11.648 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:11.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:11.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:11.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:11.649 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:11.650 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:11.651 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:11.651 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:11.653 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:11.653 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.653 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.653 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:11.654 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:11.654 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:11.654 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:11.655 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:11.655 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.655 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.656 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:11.657 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:11.658 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: test_activate_osd_skip_benchmark: run_mon td/ceph-helpers a 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:11.659 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:11.660 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.680 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:11.681 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:51:11.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:11.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:11.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:11.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:11.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:11.709 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:11.710 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:11.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:11.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:11.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:11.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:11.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.767 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.768 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:11.768 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:11.768 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:930: test_activate_osd_skip_benchmark: run_mgr td/ceph-helpers x 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:11.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:11.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:11.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:11.934 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:11.935 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:11.935 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.935 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:11.935 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:11.936 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:11.956 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:933: test_activate_osd_skip_benchmark: run_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:11.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:11.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:11.959 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:11.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dcf56df2-7f09-4e9d-a253-34a583f36204 2026-02-17T04:51:11.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dcf56df2-7f09-4e9d-a253-34a583f36204' 2026-02-17T04:51:11.960 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 dcf56df2-7f09-4e9d-a253-34a583f36204 2026-02-17T04:51:11.961 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:11.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC/85NpXaOyORAA4pAAotS/S/70f4cxPEoK7w== 2026-02-17T04:51:11.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC/85NpXaOyORAA4pAAotS/S/70f4cxPEoK7w=="}' 2026-02-17T04:51:11.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dcf56df2-7f09-4e9d-a253-34a583f36204 -i td/ceph-helpers/0/new.json 2026-02-17T04:51:12.086 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:12.093 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:12.095 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true --mkfs --key AQC/85NpXaOyORAA4pAAotS/S/70f4cxPEoK7w== --osd-uuid dcf56df2-7f09-4e9d-a253-34a583f36204 2026-02-17T04:51:12.111 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:12.106+0000 7fa639499a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:12.113 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:12.110+0000 7fa639499a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:12.114 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:12.110+0000 7fa639499a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:12.114 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:12.110+0000 7fa639499a40 -1 bdev(0x55fe82816400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:12.115 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:12.110+0000 7fa639499a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:12.904 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:12.905 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:12.905 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:12.906 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:12.906 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:13.024 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:13.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:13.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T04:51:13.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:13.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:13.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:13.051 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:13.046+0000 7f4a27f7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:13.053 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:13.050+0000 7f4a27f7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:13.055 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:13.050+0000 7f4a27f7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:13.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:13.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:13.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:13.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:13.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:13.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:13.134 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:13.134 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:13.134 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:13.134 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:13.242 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:13.454 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:13.450+0000 7f4a27f7aa40 -1 Falling back to public interface 2026-02-17T04:51:14.244 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:14.244 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:14.244 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:14.245 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:14.245 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:14.245 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:14.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:14.458 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:14.453+0000 7f4a27f7aa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:15.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:15.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:15.350 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:15.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:15.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:15.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:15.563 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2739051837,v1:127.0.0.1:6801/2739051837] [v2:127.0.0.1:6802/2739051837,v1:127.0.0.1:6803/2739051837] exists,up dcf56df2-7f09-4e9d-a253-34a583f36204 2026-02-17T04:51:15.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:15.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:15.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:15.573 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T04:51:15.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-17T04:51:15.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: local 'max_iops_hdd_def={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-17T04:51:15.623 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T04:51:15.623 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:15.623 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:15.623 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:15.623 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:15.623 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:15.624 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:15.624 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T04:51:15.624 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-17T04:51:15.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: local 'max_iops_ssd_def={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-17T04:51:15.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:940: test_activate_osd_skip_benchmark: kill_daemons td/ceph-helpers TERM osd 2026-02-17T04:51:15.687 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:15.687 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:15.687 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:15.688 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:15.688 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:15.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:15.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:941: test_activate_osd_skip_benchmark: ceph osd down 0 2026-02-17T04:51:16.207 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 is already down. 2026-02-17T04:51:16.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:942: test_activate_osd_skip_benchmark: wait_for_osd down 0 2026-02-17T04:51:16.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-17T04:51:16.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:16.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:16.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:16.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:16.217 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:16.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:16.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:16.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-17T04:51:16.448 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 7 last_clean_interval [0,0) [v2:127.0.0.1:6800/2739051837,v1:127.0.0.1:6801/2739051837] [v2:127.0.0.1:6802/2739051837,v1:127.0.0.1:6803/2739051837] exists dcf56df2-7f09-4e9d-a253-34a583f36204 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:946: test_activate_osd_skip_benchmark: activate_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T04:51:16.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:16.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:16.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:16.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:16.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T04:51:16.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-17T04:51:16.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:16.454 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:16.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T04:51:16.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T04:51:16.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T04:51:16.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T04:51:16.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T04:51:16.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T04:51:16.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T04:51:16.470 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:16.465+0000 7f5822404a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:16.471 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:16.465+0000 7f5822404a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:16.472 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:16.465+0000 7f5822404a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T04:51:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:16.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:16.665 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:16.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:16.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:16.666 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:16.878 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:16.873+0000 7f5822404a40 -1 Falling back to public interface 2026-02-17T04:51:16.888 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:17.485 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:17.481+0000 7f5822404a40 -1 osd.0 6 log_to_monitors true 2026-02-17T04:51:17.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:17.890 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:17.890 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:17.891 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:17.891 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:17.891 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:18.105 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 7 last_clean_interval [5,6) [v2:127.0.0.1:6800/1258281262,v1:127.0.0.1:6801/1258281262] [v2:127.0.0.1:6802/1258281262,v1:127.0.0.1:6803/1258281262] exists,up dcf56df2-7f09-4e9d-a253-34a583f36204 2026-02-17T04:51:18.105 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:18.105 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:18.105 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:18.106 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T04:51:18.106 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:18.106 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:18.106 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:18.107 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.107 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.107 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:18.107 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T04:51:18.107 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-17T04:51:18.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: local 'max_iops_hdd_after_boot={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-17T04:51:18.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T04:51:18.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:18.168 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:18.168 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:18.168 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.168 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.169 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:18.169 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T04:51:18.169 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-17T04:51:18.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: local 'max_iops_ssd_after_boot={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-17T04:51:18.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:953: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' = '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-17T04:51:18.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:954: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' = '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-17T04:51:18.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:956: test_activate_osd_skip_benchmark: teardown td/ceph-helpers 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:18.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:18.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:18.338 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:18.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:18.339 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:18.340 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:18.340 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:18.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:18.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:18.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:18.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:18.342 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:18.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:18.344 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:18.344 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:18.349 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:18.349 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.349 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.349 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_destroy_osd td/ceph-helpers 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:787: test_destroy_osd: local dir=td/ceph-helpers 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:789: test_destroy_osd: setup td/ceph-helpers 2026-02-17T04:51:18.350 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:18.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:18.353 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:18.354 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:18.354 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:18.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:18.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:18.355 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:18.356 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:18.357 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:18.357 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:18.357 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:18.358 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:18.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:18.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:18.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:18.361 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:18.361 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.361 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.361 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:18.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:18.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:18.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:18.364 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:18.364 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.364 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.364 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:18.365 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:790: test_destroy_osd: run_mon td/ceph-helpers a 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:18.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:51:18.387 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:18.387 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:18.387 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:18.388 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:18.388 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.388 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.388 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:18.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:51:18.417 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:18.418 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:18.419 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:18.419 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.419 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.419 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:18.419 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:18.419 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:18.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:18.476 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:18.476 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.476 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.476 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:18.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:18.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:18.531 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:791: test_destroy_osd: run_mgr td/ceph-helpers x 2026-02-17T04:51:18.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:18.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:18.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:18.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:18.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:18.532 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:18.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:18.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:18.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:18.646 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:18.646 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.646 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:18.646 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:18.647 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:18.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:792: test_destroy_osd: run_osd td/ceph-helpers 0 2026-02-17T04:51:18.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:18.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:18.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:18.670 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:18.671 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:18.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:18.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:18.672 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:18.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6debd176-55ce-4c7f-ae6c-eb71cbfa1c5c 2026-02-17T04:51:18.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6debd176-55ce-4c7f-ae6c-eb71cbfa1c5c' 2026-02-17T04:51:18.673 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 6debd176-55ce-4c7f-ae6c-eb71cbfa1c5c 2026-02-17T04:51:18.673 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:18.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDG85Np3k+JKBAAmqFN729dvqQ+kDmofJkHkQ== 2026-02-17T04:51:18.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDG85Np3k+JKBAAmqFN729dvqQ+kDmofJkHkQ=="}' 2026-02-17T04:51:18.682 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6debd176-55ce-4c7f-ae6c-eb71cbfa1c5c -i td/ceph-helpers/0/new.json 2026-02-17T04:51:18.793 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:18.800 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:18.801 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDG85Np3k+JKBAAmqFN729dvqQ+kDmofJkHkQ== --osd-uuid 6debd176-55ce-4c7f-ae6c-eb71cbfa1c5c 2026-02-17T04:51:18.818 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:18.813+0000 7f9c9f89ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:18.819 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:18.813+0000 7f9c9f89ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:18.820 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:18.813+0000 7f9c9f89ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:18.821 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:18.817+0000 7f9c9f89ca40 -1 bdev(0x564905fd0400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:18.821 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:18.817+0000 7f9c9f89ca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:19.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:19.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:19.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:19.713 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:19.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:19.827 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:19.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:19.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:19.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:19.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:19.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:19.853 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:19.849+0000 7f2da8b1aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:19.855 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:19.849+0000 7f2da8b1aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:19.857 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:19.853+0000 7f2da8b1aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:19.940 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:19.941 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:19.941 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:20.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:20.382 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:20.377+0000 7f2da8b1aa40 -1 Falling back to public interface 2026-02-17T04:51:21.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:21.053 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:21.053 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:21.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:21.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:21.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:21.263 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:21.294 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:21.289+0000 7f2da8b1aa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:22.264 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:22.264 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:22.264 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:22.265 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:22.265 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:22.265 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:22.472 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2424327497,v1:127.0.0.1:6801/2424327497] [v2:127.0.0.1:6802/2424327497,v1:127.0.0.1:6803/2424327497] exists,up 6debd176-55ce-4c7f-ae6c-eb71cbfa1c5c 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:793: test_destroy_osd: destroy_osd td/ceph-helpers 0 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/ceph-helpers 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=0 2026-02-17T04:51:22.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.0 2026-02-17T04:51:22.720 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 is already out. 2026-02-17T04:51:22.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-17T04:51:22.729 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:22.729 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:22.730 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:22.730 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:22.730 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:23.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:23.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.0 2026-02-17T04:51:23.256 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 is already down. 2026-02-17T04:51:23.266 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.0 --yes-i-really-mean-it 2026-02-17T04:51:23.492 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 does not exist 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/ceph-helpers/0 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers/0 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers/0 KILL 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:23.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:23.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:23.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:23.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:23.506 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:23.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:23.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:23.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:23.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:23.508 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:23.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:23.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:23.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:23.510 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:23.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:23.512 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:23.512 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers/0 2026-02-17T04:51:23.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:23.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.515 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:23.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:23.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:23.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/ceph-helpers/0 2026-02-17T04:51:23.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: ceph osd dump 2026-02-17T04:51:23.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: grep 'osd. ' 2026-02-17T04:51:23.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:795: test_destroy_osd: teardown td/ceph-helpers 2026-02-17T04:51:23.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:23.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:23.731 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:23.732 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:23.732 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:23.732 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:23.732 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:23.732 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:23.840 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:23.840 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:23.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:23.841 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:23.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:23.843 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:23.843 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:23.844 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:23.844 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:23.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:23.845 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:23.845 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:23.847 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:23.847 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:23.849 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:23.849 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.849 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.849 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_display_logs td/ceph-helpers 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2158: test_display_logs: local dir=td/ceph-helpers 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2160: test_display_logs: setup td/ceph-helpers 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:23.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:23.853 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:23.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:23.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:23.854 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:23.855 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:23.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:23.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:23.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:23.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:23.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:23.858 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:23.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:23.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:23.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:23.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:23.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:23.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:23.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:23.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:23.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:23.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:23.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.864 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:23.866 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2161: test_display_logs: run_mon td/ceph-helpers a 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:23.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:23.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:51:23.916 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:23.917 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:23.917 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:23.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:23.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:23.918 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:23.919 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:23.919 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:23.919 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:23.919 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:23.919 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.920 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.920 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:23.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:23.920 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:23.983 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:23.984 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:23.984 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:23.984 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:23.985 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:23.985 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:23.985 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:23.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:23.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:24.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2162: test_display_logs: kill_daemons td/ceph-helpers 2026-02-17T04:51:24.041 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:24.041 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:24.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:24.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:24.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:24.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:24.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2163: test_display_logs: display_logs td/ceph-helpers 2026-02-17T04:51:24.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2148: display_logs: local dir=td/ceph-helpers 2026-02-17T04:51:24.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: display_logs: find td/ceph-helpers -maxdepth 1 -name '*.log' 2026-02-17T04:51:24.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-17T04:51:24.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2152: display_logs: echo '======================= td/ceph-helpers/mon.a.log' 2026-02-17T04:51:24.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2153: display_logs: cat td/ceph-helpers/mon.a.log 2026-02-17T04:51:24.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-17T04:51:24.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2164: test_display_logs: grep --quiet mon.a.log td/ceph-helpers/log.out 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2165: test_display_logs: teardown td/ceph-helpers 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:24.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:24.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:24.154 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:24.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:24.155 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:24.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:24.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:24.157 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:24.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:24.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:24.159 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.160 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:24.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:24.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:24.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:24.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:24.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:24.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:24.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_erasure_code_plugin_exists td/ceph-helpers 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: test_erasure_code_plugin_exists: local dir=td/ceph-helpers 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: test_erasure_code_plugin_exists: setup td/ceph-helpers 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:24.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:24.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:24.167 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:24.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:24.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:24.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:24.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:24.170 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:24.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:24.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:24.171 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:24.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:24.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:24.175 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:24.175 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.175 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:24.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:24.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:24.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:24.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:24.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:24.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:24.180 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:24.180 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:24.180 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:24.180 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: test_erasure_code_plugin_exists: run_mon td/ceph-helpers a 2026-02-17T04:51:24.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:24.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:24.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:24.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:24.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:24.181 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:51:24.200 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:24.200 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:24.200 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:24.201 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:24.201 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.201 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.201 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:24.201 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:51:24.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:24.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:24.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:24.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:24.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:24.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:24.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:24.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:24.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:24.230 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:24.230 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.231 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.231 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:24.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:24.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:24.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:24.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:24.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:24.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:24.288 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:24.288 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: test_erasure_code_plugin_exists: run_mgr td/ceph-helpers x 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:24.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:24.454 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:24.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:24.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:24.455 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:24.455 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.455 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.455 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:24.456 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:24.456 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:24.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: test_erasure_code_plugin_exists: erasure_code_plugin_exists jerasure 2026-02-17T04:51:24.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=jerasure 2026-02-17T04:51:24.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-17T04:51:24.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-17T04:51:24.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-17T04:51:24.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-17T04:51:24.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-17T04:51:24.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='jerasure.*No such file' 2026-02-17T04:51:24.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=jerasure 2026-02-17T04:51:24.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-17T04:51:24.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-17T04:51:24.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-17T04:51:24.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-17T04:51:24.752 INFO:tasks.workunit.client.0.trial105.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-17T04:51:24.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: test_erasure_code_plugin_exists: erasure_code_plugin_exists FAKE 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=FAKE 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-17T04:51:24.761 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-17T04:51:24.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='FAKE.*No such file' 2026-02-17T04:51:24.762 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=FAKE 2026-02-17T04:51:24.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s='Error EIO: load dlopen(/usr/lib/ceph/erasure-code/libec_FAKE.so): /usr/lib/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory' 2026-02-17T04:51:24.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=5 2026-02-17T04:51:24.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 5 -eq 0 ']' 2026-02-17T04:51:24.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: echo Error EIO: load 'dlopen(/usr/lib/ceph/erasure-code/libec_FAKE.so):' /usr/lib/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory 2026-02-17T04:51:24.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: grep --quiet 'FAKE.*No such file' 2026-02-17T04:51:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2119: erasure_code_plugin_exists: status=1 2026-02-17T04:51:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 1 2026-02-17T04:51:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: test_erasure_code_plugin_exists: teardown td/ceph-helpers 2026-02-17T04:51:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:24.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:24.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:24.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:24.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:24.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:24.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:24.971 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:24.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:24.972 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:24.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:24.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:24.974 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:24.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:24.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:24.975 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:24.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:24.977 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:24.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:24.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.979 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.979 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:24.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:24.981 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_expect_failure td/ceph-helpers 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: test_expect_failure: local dir=td/ceph-helpers 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2039: test_expect_failure: setup td/ceph-helpers 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:24.982 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:24.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:24.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:24.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:24.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:24.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:24.984 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:24.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:24.986 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:24.986 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:24.986 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:24.987 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.987 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:24.988 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:24.988 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:24.989 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:24.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:24.990 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:24.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:24.992 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.992 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.992 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:24.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:24.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:24.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:24.994 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:24.994 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:24.994 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:24.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:24.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:24.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:24.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:24.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:24.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2040: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 1' 2026-02-17T04:51:24.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-17T04:51:24.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:51:24.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-17T04:51:24.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:51:24.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:51:24.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 1' 2026-02-17T04:51:24.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T04:51:24.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T04:51:24.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T04:51:25.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T04:51:25.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2042: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 0' 2026-02-17T04:51:25.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-17T04:51:25.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:51:25.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-17T04:51:25.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:51:25.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:51:25.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 0' 2026-02-17T04:51:25.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2023: expect_failure: success=true 2026-02-17T04:51:25.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: true 2026-02-17T04:51:25.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-17T04:51:25.002 INFO:tasks.workunit.client.0.trial105.stderr:FAIL 2026-02-17T04:51:25.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-17T04:51:25.002 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2043: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo UNEXPECTED ; exit 1' 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T04:51:25.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo UNEXPECTED ; exit 1' 2026-02-17T04:51:25.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T04:51:25.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T04:51:25.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T04:51:25.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-17T04:51:25.006 INFO:tasks.workunit.client.0.trial105.stderr:UNEXPECTED 2026-02-17T04:51:25.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-17T04:51:25.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T04:51:25.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: test_expect_failure: teardown td/ceph-helpers 2026-02-17T04:51:25.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:25.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:25.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:25.009 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:25.009 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:25.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:25.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:25.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:25.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:25.011 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:25.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:25.012 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:25.013 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:25.013 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:25.014 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:25.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:25.015 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:25.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.016 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:25.017 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:25.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:25.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:25.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:25.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:25.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:25.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_file_with_random_data td/ceph-helpers 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2350: test_file_with_random_data: local dir=td/ceph-helpers 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2352: test_file_with_random_data: setup td/ceph-helpers 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:25.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:25.023 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:25.023 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:25.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:25.024 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:25.025 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:25.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:25.025 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:25.026 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:25.027 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.027 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:25.027 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:25.028 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:25.029 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:25.030 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:25.031 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:25.031 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.031 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.031 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:25.032 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:25.032 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:25.032 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:25.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:25.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.033 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:25.034 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:25.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:25.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:25.035 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:25.035 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: file_with_random_data 4000 2026-02-17T04:51:25.035 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2343: file_with_random_data: local size_bytes=4000 2026-02-17T04:51:25.036 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: mktemp 2026-02-17T04:51:25.037 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local file=/tmp/tmp.uLAhYntCRk 2026-02-17T04:51:25.037 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.uLAhYntCRk bs=4000 count=1 2026-02-17T04:51:25.038 INFO:tasks.workunit.client.0.trial105.stderr:1+0 records in 2026-02-17T04:51:25.038 INFO:tasks.workunit.client.0.trial105.stderr:1+0 records out 2026-02-17T04:51:25.038 INFO:tasks.workunit.client.0.trial105.stderr:4000 bytes (4.0 kB, 3.9 KiB) copied, 0.000103 s, 38.8 MB/s 2026-02-17T04:51:25.038 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: printf %s /tmp/tmp.uLAhYntCRk 2026-02-17T04:51:25.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: local file=/tmp/tmp.uLAhYntCRk 2026-02-17T04:51:25.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: test -f /tmp/tmp.uLAhYntCRk 2026-02-17T04:51:25.039 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: stat -c %s /tmp/tmp.uLAhYntCRk 2026-02-17T04:51:25.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: test 4000 = 4000 2026-02-17T04:51:25.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: rm /tmp/tmp.uLAhYntCRk 2026-02-17T04:51:25.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2357: test_file_with_random_data: teardown td/ceph-helpers 2026-02-17T04:51:25.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:25.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:25.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:25.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:25.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:25.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:25.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:25.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:25.043 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:25.044 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:25.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:25.045 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:25.046 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:25.046 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:25.046 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:25.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:25.048 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:25.048 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.049 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:25.050 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:25.050 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:25.051 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:25.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:25.053 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_flush_pg_stats td/ceph-helpers 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2280: test_flush_pg_stats: local dir=td/ceph-helpers 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: test_flush_pg_stats: setup td/ceph-helpers 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:25.054 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:25.055 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:25.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:25.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:25.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:25.055 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:25.055 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:25.056 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:25.056 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:25.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:25.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:25.058 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:25.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:25.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:25.059 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:25.060 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:25.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:25.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:25.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:25.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.063 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:25.064 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:25.064 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:25.064 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:25.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:25.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:25.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:25.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:25.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:25.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:25.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2283: test_flush_pg_stats: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:25.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:25.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:25.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:25.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:25.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:25.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:25.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:25.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:25.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:25.090 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:25.090 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.090 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.090 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:25.090 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:25.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:25.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:25.121 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:25.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:25.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:25.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:25.122 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:25.123 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:25.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:25.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:25.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:25.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:25.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:25.180 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:25.180 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:25.180 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:25.181 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:25.181 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.182 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.182 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:25.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:25.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:25.237 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: test_flush_pg_stats: run_mgr td/ceph-helpers x 2026-02-17T04:51:25.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:25.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:25.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:25.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:25.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:25.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:25.354 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:25.354 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:25.354 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:25.355 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:25.355 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.355 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:25.355 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:25.356 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: test_flush_pg_stats: run_osd td/ceph-helpers 0 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:25.375 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:25.376 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:25.377 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:25.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:25.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:25.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:25.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:25.379 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:25.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f9183571-2911-46e6-a0db-dc863d253056 2026-02-17T04:51:25.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f9183571-2911-46e6-a0db-dc863d253056' 2026-02-17T04:51:25.379 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 f9183571-2911-46e6-a0db-dc863d253056 2026-02-17T04:51:25.379 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:25.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDN85Npx3sGFxAAsh9T9zh6yTkr9fBCFCu97w== 2026-02-17T04:51:25.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDN85Npx3sGFxAAsh9T9zh6yTkr9fBCFCu97w=="}' 2026-02-17T04:51:25.388 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f9183571-2911-46e6-a0db-dc863d253056 -i td/ceph-helpers/0/new.json 2026-02-17T04:51:25.501 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:25.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:25.510 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDN85Npx3sGFxAAsh9T9zh6yTkr9fBCFCu97w== --osd-uuid f9183571-2911-46e6-a0db-dc863d253056 2026-02-17T04:51:25.526 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:25.521+0000 7f6a95dcba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:25.529 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:25.521+0000 7f6a95dcba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:25.530 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:25.525+0000 7f6a95dcba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:25.530 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:25.525+0000 7f6a95dcba40 -1 bdev(0x55997e848400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:25.530 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:25.525+0000 7f6a95dcba40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:26.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:26.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:26.224 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:26.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:26.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:26.337 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:26.338 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:26.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:26.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:26.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:26.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:26.353 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:26.349+0000 7fbfd59fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:26.355 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:26.349+0000 7fbfd59fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:26.355 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:26.349+0000 7fbfd59fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:26.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:26.454 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:26.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:26.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:26.560 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:26.758 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:26.753+0000 7fbfd59fca40 -1 Falling back to public interface 2026-02-17T04:51:27.422 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:27.417+0000 7fbfd59fca40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:27.562 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:27.562 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:27.563 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:27.563 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:27.564 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:27.567 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:27.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:28.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:28.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:28.712 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:28.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:28.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:28.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:28.931 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3555905654,v1:127.0.0.1:6801/3555905654] [v2:127.0.0.1:6802/3555905654,v1:127.0.0.1:6803/3555905654] exists,up f9183571-2911-46e6-a0db-dc863d253056 2026-02-17T04:51:28.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:28.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:28.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:28.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: test_flush_pg_stats: create_rbd_pool 2026-02-17T04:51:28.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:51:29.137 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:51:29.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:51:29.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:51:29.388 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:51:29.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:51:30.401 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:51:30.678 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: test_flush_pg_stats: rados -p rbd put obj /etc/group 2026-02-17T04:51:30.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: test_flush_pg_stats: flush_pg_stats 2026-02-17T04:51:30.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:51:30.697 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:51:30.917 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:51:30.917 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:51:30.917 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:51:30.917 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:51:30.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:51:30.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:51:30.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:51:30.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:51:30.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:51:30.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:51:30.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:51:30.999 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:51:31.000 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:51:31.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:51:31.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:51:31.001 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:51:31.001 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:51:31.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:51:31.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:51:32.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:51:32.217 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:51:32.428 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:51:32.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: test_flush_pg_stats: local 'jq_filter=.pools | .[] | select(.name == "rbd") | .stats' 2026-02-17T04:51:32.429 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: ceph df detail --format=json 2026-02-17T04:51:32.429 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored' 2026-02-17T04:51:32.709 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: stored=8348 2026-02-17T04:51:32.710 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: ceph df detail --format=json 2026-02-17T04:51:32.710 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored_raw' 2026-02-17T04:51:32.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: stored_raw=8348 2026-02-17T04:51:32.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: test 8348 -gt 0 2026-02-17T04:51:32.996 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2293: test_flush_pg_stats: test 8348 == 8348 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: test_flush_pg_stats: teardown td/ceph-helpers 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:32.997 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:33.105 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:33.106 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:33.107 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:33.107 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:33.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:33.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:33.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:33.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:33.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:33.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:33.110 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:33.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:33.112 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:33.113 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:33.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:33.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.118 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_config td/ceph-helpers 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: test_get_config: local dir=td/ceph-helpers 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: test_get_config: setup td/ceph-helpers 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:33.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:33.122 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:33.123 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:33.123 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:33.124 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:33.124 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:33.125 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:33.125 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:33.126 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:33.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:33.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:33.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:33.127 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:33.128 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:33.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:33.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:33.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:33.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.131 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.131 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:33.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:33.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:33.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:33.133 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:33.133 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.133 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:33.135 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1131: test_get_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:33.136 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:33.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:33.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:33.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:33.156 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:33.156 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:33.156 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:33.156 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:33.157 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.157 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.157 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:33.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:33.184 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:33.185 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:33.185 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:33.185 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:33.185 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:33.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:33.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:33.243 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:33.244 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:33.244 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.244 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.244 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:33.244 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:33.244 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:33.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: get_config mon a osd_pool_default_size 2026-02-17T04:51:33.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:33.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:33.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_pool_default_size 2026-02-17T04:51:33.298 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_pool_default_size 2026-02-17T04:51:33.299 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:33.299 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:33.300 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get osd_pool_default_size 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: test 1 = 1 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1133: test_get_config: run_mgr td/ceph-helpers x 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:33.358 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:33.359 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:33.472 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:33.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:33.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: test_get_config: run_osd td/ceph-helpers 0 --osd_max_scrubs=3 2026-02-17T04:51:33.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:33.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:33.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:33.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:33.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:33.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:33.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_max_scrubs=3 2026-02-17T04:51:33.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:33.497 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:33.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5a7e4776-38a1-48b8-8ebe-3e8adaf95479 2026-02-17T04:51:33.497 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 5a7e4776-38a1-48b8-8ebe-3e8adaf95479 2026-02-17T04:51:33.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5a7e4776-38a1-48b8-8ebe-3e8adaf95479' 2026-02-17T04:51:33.498 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:33.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDV85NpDVcbHhAA16tn6qgupJbBeqJHbiNjlg== 2026-02-17T04:51:33.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDV85NpDVcbHhAA16tn6qgupJbBeqJHbiNjlg=="}' 2026-02-17T04:51:33.507 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5a7e4776-38a1-48b8-8ebe-3e8adaf95479 -i td/ceph-helpers/0/new.json 2026-02-17T04:51:33.620 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:33.628 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:33.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 --mkfs --key AQDV85NpDVcbHhAA16tn6qgupJbBeqJHbiNjlg== --osd-uuid 5a7e4776-38a1-48b8-8ebe-3e8adaf95479 2026-02-17T04:51:33.645 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:33.637+0000 7f3f4cbb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:33.647 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:33.641+0000 7f3f4cbb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:33.648 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:33.641+0000 7f3f4cbb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:33.648 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:33.641+0000 7f3f4cbb7a40 -1 bdev(0x55eeda8dc400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:33.648 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:33.641+0000 7f3f4cbb7a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:34.417 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:34.417 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:34.418 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:34.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:34.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:34.538 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:34.538 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:34.538 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:34.538 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 2026-02-17T04:51:34.538 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:34.538 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:34.554 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:34.549+0000 7f227a9e3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:34.555 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:34.549+0000 7f227a9e3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:34.557 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:34.553+0000 7f227a9e3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:34.662 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:34.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:34.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:34.770 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:35.050 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:35.045+0000 7f227a9e3a40 -1 Falling back to public interface 2026-02-17T04:51:35.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:35.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:35.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:35.772 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:35.773 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:35.773 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:35.879 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:36.053 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:36.049+0000 7f227a9e3a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:36.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:36.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:36.880 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:36.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:36.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:36.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:37.091 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/171558036,v1:127.0.0.1:6801/171558036] [v2:127.0.0.1:6802/171558036,v1:127.0.0.1:6803/171558036] exists,up 5a7e4776-38a1-48b8-8ebe-3e8adaf95479 2026-02-17T04:51:37.092 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:37.092 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:37.092 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: get_config osd 0 osd_max_scrubs 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_max_scrubs 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_max_scrubs 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:51:37.093 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:51:37.094 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:37.094 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.094 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.094 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:51:37.095 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:37.095 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_max_scrubs 2026-02-17T04:51:37.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: test 3 = 3 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1136: test_get_config: teardown td/ceph-helpers 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:37.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:37.264 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:37.264 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:37.265 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:37.266 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:37.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:37.267 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:37.267 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:37.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:37.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:37.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:37.271 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:37.271 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:37.274 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:37.275 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.275 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.275 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:37.277 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:37.277 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_is_making_recovery_progress td/ceph-helpers 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: test_get_is_making_recovery_progress: local dir=td/ceph-helpers 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: test_get_is_making_recovery_progress: setup td/ceph-helpers 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:37.278 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:37.279 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:37.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:37.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:37.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:37.279 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:37.280 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:37.280 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:37.280 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:37.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:37.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:37.282 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:37.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:37.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:37.283 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.284 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:37.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:37.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:37.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:37.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.287 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.287 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:37.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:37.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:37.288 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:37.290 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:37.290 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.290 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.290 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:37.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1338: test_get_is_making_recovery_progress: run_mon td/ceph-helpers a 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:37.293 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:37.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:37.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:37.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:51:37.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:37.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:37.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:37.314 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:37.314 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.314 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.315 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:37.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:51:37.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:37.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:37.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:37.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:37.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:37.353 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:37.353 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:37.353 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:37.353 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:37.353 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:37.354 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.354 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.354 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:37.354 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:37.354 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:37.408 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:37.409 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:37.409 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:37.409 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:37.409 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:37.410 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:37.410 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:37.410 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:37.410 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:37.410 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.411 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.411 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:37.411 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:37.411 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: test_get_is_making_recovery_progress: run_mgr td/ceph-helpers x 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:37.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:37.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:37.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:37.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:37.575 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:37.575 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.575 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.575 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:37.575 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:37.576 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:37.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: test_get_is_making_recovery_progress: get_is_making_recovery_progress 2026-02-17T04:51:37.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-17T04:51:37.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T04:51:37.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T04:51:37.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T04:51:37.595 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-17T04:51:37.595 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T04:51:37.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-17T04:51:37.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-17T04:51:37.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1341: test_get_is_making_recovery_progress: teardown td/ceph-helpers 2026-02-17T04:51:37.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:37.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:37.815 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:37.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:37.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:37.816 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:37.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:37.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:37.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:37.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:37.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:37.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:37.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:37.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:37.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:37.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.824 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:37.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:37.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_last_scrub_stamp td/ceph-helpers 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1524: test_get_last_scrub_stamp: local dir=td/ceph-helpers 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: test_get_last_scrub_stamp: setup td/ceph-helpers 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:37.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:37.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:37.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:37.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:37.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:37.829 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:37.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:37.830 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:37.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:37.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:37.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:37.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:37.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:37.833 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:37.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:37.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:37.835 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:37.836 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:37.836 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.836 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:37.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:37.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:37.837 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:37.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:37.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.838 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:37.840 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:37.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:37.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:37.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:37.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: test_get_last_scrub_stamp: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:37.841 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:37.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:37.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:37.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:37.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:37.842 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.863 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:37.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:37.893 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:37.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:37.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:37.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:37.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:37.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:37.895 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:37.895 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:37.896 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:37.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:37.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:37.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:37.951 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:37.951 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:37.952 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:37.953 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:38.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: test_get_last_scrub_stamp: run_mgr td/ceph-helpers x 2026-02-17T04:51:38.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:38.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:38.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:38.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:38.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:38.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:38.117 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:38.117 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:38.117 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:38.117 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:38.117 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:38.118 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:38.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:38.118 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:38.119 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:38.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: test_get_last_scrub_stamp: run_osd td/ceph-helpers 0 2026-02-17T04:51:38.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:38.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:38.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:38.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:38.141 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:38.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:38.143 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:38.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=95a74d70-fcb1-45b9-bb72-b05fb27b5c00 2026-02-17T04:51:38.144 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 95a74d70-fcb1-45b9-bb72-b05fb27b5c00 2026-02-17T04:51:38.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 95a74d70-fcb1-45b9-bb72-b05fb27b5c00' 2026-02-17T04:51:38.145 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:38.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDa85NpG0dbCRAAjhs+MI7DXXO3p41GXxvicw== 2026-02-17T04:51:38.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDa85NpG0dbCRAAjhs+MI7DXXO3p41GXxvicw=="}' 2026-02-17T04:51:38.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 95a74d70-fcb1-45b9-bb72-b05fb27b5c00 -i td/ceph-helpers/0/new.json 2026-02-17T04:51:38.274 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:38.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:38.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDa85NpG0dbCRAAjhs+MI7DXXO3p41GXxvicw== --osd-uuid 95a74d70-fcb1-45b9-bb72-b05fb27b5c00 2026-02-17T04:51:38.299 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:38.293+0000 7faae5b52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:38.301 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:38.297+0000 7faae5b52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:38.302 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:38.297+0000 7faae5b52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:38.302 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:38.297+0000 7faae5b52a40 -1 bdev(0x55606408e400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:38.302 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:38.297+0000 7faae5b52a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:39.072 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:39.073 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:39.073 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:39.074 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:39.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:39.193 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:39.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:39.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:39.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:39.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:39.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:39.219 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:39.213+0000 7f9b14205a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:39.220 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:39.213+0000 7f9b14205a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:39.222 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:39.217+0000 7f9b14205a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:39.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:39.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:39.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:39.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:39.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:39.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:39.302 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:39.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:39.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:39.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:39.417 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:39.735 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:39.729+0000 7f9b14205a40 -1 Falling back to public interface 2026-02-17T04:51:40.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:40.418 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:40.418 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:40.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:40.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:40.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:40.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:40.718 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:40.713+0000 7f9b14205a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:41.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:41.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:41.530 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:41.531 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:41.531 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:41.531 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:41.743 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2157931190,v1:127.0.0.1:6801/2157931190] [v2:127.0.0.1:6802/2157931190,v1:127.0.0.1:6803/2157931190] exists,up 95a74d70-fcb1-45b9-bb72-b05fb27b5c00 2026-02-17T04:51:41.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:41.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:41.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:41.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1530: test_get_last_scrub_stamp: create_rbd_pool 2026-02-17T04:51:41.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:51:41.950 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:51:41.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:51:41.962 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:51:42.187 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:51:42.197 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:51:43.198 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:51:43.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1531: test_get_last_scrub_stamp: wait_for_clean 2026-02-17T04:51:43.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:51:43.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:51:43.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:51:43.500 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:51:43.501 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:51:43.501 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:51:43.501 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:51:43.501 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:51:43.501 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:51:43.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:51:43.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:51:43.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:51:43.603 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:51:43.603 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:51:43.603 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:51:43.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:51:43.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:51:43.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:51:43.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:51:43.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:51:43.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:51:43.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:51:43.898 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:51:43.898 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:51:43.899 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:51:43.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:51:43.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:51:43.900 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:51:43.901 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:51:43.902 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:51:43.902 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:51:43.902 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:51:44.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:51:44.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:51:45.121 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:51:45.122 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:51:45.335 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:51:45.336 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:51:45.336 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:51:45.336 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:51:45.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:51:45.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:51:45.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:51:45.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:51:45.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:51:45.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:51:45.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:51:45.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:51:45.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:51:45.839 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:51:45.840 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:51:45.840 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:51:46.139 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:51:46.139 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:51:46.139 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:51:46.139 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: get_last_scrub_stamp 1.0 2026-02-17T04:51:46.140 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:51:46.140 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:51:46.140 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:51:46.140 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:51:46.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: stamp=2026-02-17T04:51:42.133209+0000 2026-02-17T04:51:46.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1533: test_get_last_scrub_stamp: test -n 2026-02-17T04:51:42.133209+0000 2026-02-17T04:51:46.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1534: test_get_last_scrub_stamp: teardown td/ceph-helpers 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:46.352 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:46.460 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:46.461 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:46.461 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:46.461 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:46.463 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:46.463 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:46.463 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:46.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:46.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:46.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:46.465 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:46.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:46.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:46.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:46.472 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:46.472 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.472 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_not_primary td/ceph-helpers 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: test_get_not_primary: local dir=td/ceph-helpers 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1231: test_get_not_primary: setup td/ceph-helpers 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:46.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:46.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:46.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:46.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:46.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:46.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:46.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:46.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:46.477 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:46.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:46.478 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:46.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:46.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:46.479 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:46.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:46.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:46.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:46.481 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:46.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:46.482 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:46.483 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:46.484 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:46.484 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.484 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.484 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:46.485 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:46.485 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:46.485 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:46.487 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:46.487 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.487 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:46.488 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: test_get_not_primary: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:46.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:46.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:46.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:46.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-17T04:51:46.510 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:46.510 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:46.510 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:46.511 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:46.511 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.511 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.511 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:46.511 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T04:51:46.540 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:46.541 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:46.541 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:46.541 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:46.541 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:46.543 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:46.543 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:46.543 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:46.543 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:46.543 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:46.543 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.544 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.544 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:46.544 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:46.544 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:46.600 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:46.601 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:46.601 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.602 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.602 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:46.602 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:46.602 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: test_get_not_primary: run_mgr td/ceph-helpers x 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:46.657 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:46.770 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:46.770 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:46.770 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:46.771 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:46.771 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.771 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.771 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:46.772 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:46.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:46.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: test_get_not_primary: run_osd td/ceph-helpers 0 2026-02-17T04:51:46.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:46.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:46.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:46.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=36182613-a9fe-4a1d-a42d-eaf3aae79da8 2026-02-17T04:51:46.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 36182613-a9fe-4a1d-a42d-eaf3aae79da8' 2026-02-17T04:51:46.798 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 36182613-a9fe-4a1d-a42d-eaf3aae79da8 2026-02-17T04:51:46.798 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:46.806 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDi85Np7aX4LxAAuIBjDuM8aJi6y1cR6VE6lA== 2026-02-17T04:51:46.806 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDi85Np7aX4LxAAuIBjDuM8aJi6y1cR6VE6lA=="}' 2026-02-17T04:51:46.806 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 36182613-a9fe-4a1d-a42d-eaf3aae79da8 -i td/ceph-helpers/0/new.json 2026-02-17T04:51:46.916 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:46.924 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:46.925 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDi85Np7aX4LxAAuIBjDuM8aJi6y1cR6VE6lA== --osd-uuid 36182613-a9fe-4a1d-a42d-eaf3aae79da8 2026-02-17T04:51:46.942 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:46.937+0000 7fc730b1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:46.944 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:46.937+0000 7fc730b1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:46.945 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:46.941+0000 7fc730b1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:46.945 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:46.941+0000 7fc730b1ca40 -1 bdev(0x5587b8d08400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:46.946 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:46.941+0000 7fc730b1ca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:51:47.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:51:47.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:47.642 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:51:47.642 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:51:47.642 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:47.754 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:51:47.754 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:51:47.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:47.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:47.756 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:47.756 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:47.770 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:47.765+0000 7f20cbfe3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:47.771 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:47.765+0000 7f20cbfe3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:47.773 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:47.765+0000 7f20cbfe3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:47.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:51:47.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:47.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:51:47.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:47.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:47.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:47.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:47.862 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:47.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:47.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:47.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:48.315 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:48.309+0000 7f20cbfe3a40 -1 Falling back to public interface 2026-02-17T04:51:48.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:48.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:48.971 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:48.972 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:48.973 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:48.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:49.117 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:49.309 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:49.301+0000 7f20cbfe3a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:51:50.119 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:50.119 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:50.119 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:51:50.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:51:50.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:50.120 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:51:50.331 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1573823212,v1:127.0.0.1:6801/1573823212] [v2:127.0.0.1:6802/1573823212,v1:127.0.0.1:6803/1573823212] exists,up 36182613-a9fe-4a1d-a42d-eaf3aae79da8 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1235: test_get_not_primary: run_osd td/ceph-helpers 1 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:50.332 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:50.333 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-17T04:51:50.334 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:50.335 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=973ec32a-2988-4a5a-a619-27d3cba37e29 2026-02-17T04:51:50.335 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 973ec32a-2988-4a5a-a619-27d3cba37e29' 2026-02-17T04:51:50.335 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 973ec32a-2988-4a5a-a619-27d3cba37e29 2026-02-17T04:51:50.336 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:50.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDm85Npjc6bFBAAo3hbX7SsvinCGg6mtcl9BA== 2026-02-17T04:51:50.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDm85Npjc6bFBAAo3hbX7SsvinCGg6mtcl9BA=="}' 2026-02-17T04:51:50.347 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 973ec32a-2988-4a5a-a619-27d3cba37e29 -i td/ceph-helpers/1/new.json 2026-02-17T04:51:50.556 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:50.565 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-17T04:51:50.567 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDm85Npjc6bFBAAo3hbX7SsvinCGg6mtcl9BA== --osd-uuid 973ec32a-2988-4a5a-a619-27d3cba37e29 2026-02-17T04:51:50.583 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:50.577+0000 7f7036bc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:50.584 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:50.577+0000 7f7036bc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:50.585 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:50.581+0000 7f7036bc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:50.586 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:50.581+0000 7f7036bc8a40 -1 bdev(0x558044352400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:50.586 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:50.581+0000 7f7036bc8a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-17T04:51:51.215 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-17T04:51:51.215 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:51:51.216 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:51:51.217 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:51:51.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:51:51.519 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:51:51.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:51:51.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:51:51.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:51:51.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:51:51.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:51:51.543 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:51.536+0000 7f14b4847a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:51.545 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:51.536+0000 7f14b4847a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:51.546 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:51.540+0000 7f14b4847a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:51.728 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:51.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:51:51.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:51.729 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:51:51.891 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:51.884+0000 7f14b4847a40 -1 Falling back to public interface 2026-02-17T04:51:51.942 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:51:52.480 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:52.472+0000 7f14b4847a40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:51:52.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:51:52.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:51:52.944 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:51:52.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:51:52.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:51:52.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:51:53.153 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2729905483,v1:127.0.0.1:6811/2729905483] [v2:127.0.0.1:6812/2729905483,v1:127.0.0.1:6813/2729905483] exists,up 973ec32a-2988-4a5a-a619-27d3cba37e29 2026-02-17T04:51:53.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:51:53.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:51:53.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:51:53.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1236: test_get_not_primary: create_rbd_pool 2026-02-17T04:51:53.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:51:53.358 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:51:53.366 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:51:53.367 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:51:53.610 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:51:53.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:51:54.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:51:55.271 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1237: test_get_not_primary: wait_for_clean 2026-02-17T04:51:55.272 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:51:55.272 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:51:55.272 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:51:55.272 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:51:55.273 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:51:55.273 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:51:55.273 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:51:55.273 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:51:55.273 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:51:55.364 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:51:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:51:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:51:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:51:55.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:51:55.365 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:51:55.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:51:55.582 INFO:tasks.workunit.client.0.trial105.stderr:1' 2026-02-17T04:51:55.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:51:55.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:51:55.582 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:51:55.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-17T04:51:55.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-17T04:51:55.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T04:51:55.663 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:51:55.663 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:51:55.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-17T04:51:55.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-17T04:51:55.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-17T04:51:55.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:51:55.742 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-17T04:51:55.743 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:51:55.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:51:55.744 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-17T04:51:55.744 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:51:55.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-17T04:51:55.746 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836484 2026-02-17T04:51:55.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T04:51:55.746 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:51:55.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-17T04:51:55.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:51:56.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:51:56.971 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:51:57.190 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T04:51:57.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:51:57.191 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-17T04:51:57.191 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:51:57.192 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:51:57.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-17T04:51:57.193 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:51:57.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-17T04:51:57.195 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 42949672964 2026-02-17T04:51:57.195 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T04:51:57.195 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:51:57.412 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T04:51:57.413 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:51:57.413 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:51:57.414 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:51:57.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:51:57.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:51:57.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:51:57.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:51:57.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:51:57.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:51:57.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:51:57.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:51:57.900 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:51:57.901 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:51:57.901 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:51:57.901 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: get_primary rbd GROUP 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-17T04:51:58.182 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-17T04:51:58.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: local primary=1 2026-02-17T04:51:58.390 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: get_not_primary rbd GROUP 2026-02-17T04:51:58.390 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-17T04:51:58.390 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=GROUP 2026-02-17T04:51:58.390 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd GROUP 2026-02-17T04:51:58.391 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-17T04:51:58.391 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-17T04:51:58.391 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-17T04:51:58.391 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-17T04:51:58.611 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-17T04:51:58.612 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd GROUP 2026-02-17T04:51:58.612 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T04:51:58.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: local not_primary=0 2026-02-17T04:51:58.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1240: test_get_not_primary: test 0 '!=' 1 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1241: test_get_not_primary: test 0 = 0 -o 0 = 1 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1242: test_get_not_primary: teardown td/ceph-helpers 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:58.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:58.943 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:58.943 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:58.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:58.945 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:58.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:58.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:58.946 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:58.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:58.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:58.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:58.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:58.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:58.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:58.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:58.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:58.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:58.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:58.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:58.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:58.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:58.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:51:58.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_clean td/ceph-helpers 2026-02-17T04:51:58.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: test_get_num_active_clean: local dir=td/ceph-helpers 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: test_get_num_active_clean: setup td/ceph-helpers 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:51:58.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:51:58.961 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:51:58.961 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:51:58.962 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:51:58.963 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:51:58.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:51:58.963 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:51:58.964 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:51:58.965 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:58.965 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:51:58.965 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:51:58.966 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:51:58.966 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:51:58.968 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:51:58.968 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:51:58.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:51:58.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:58.969 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:58.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:51:58.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:51:58.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:51:58.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:51:58.972 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:51:58.972 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:58.972 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:58.972 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:51:58.974 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: test_get_num_active_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:51:58.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:51:58.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:51:58.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:51:58.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:58.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:51:58.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:58.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:58.997 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:58.998 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:58.998 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:58.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:58.998 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:51:59.028 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:51:59.029 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:51:59.030 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:59.030 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:59.030 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:51:59.030 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:51:59.031 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:59.031 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:59.031 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:59.031 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:59.032 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:59.032 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:59.032 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:59.032 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:59.032 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:51:59.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:51:59.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:51:59.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:51:59.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:51:59.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:51:59.145 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: test_get_num_active_clean: run_mgr td/ceph-helpers x 2026-02-17T04:51:59.145 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:51:59.145 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:51:59.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:51:59.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:51:59.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:51:59.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:51:59.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:51:59.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:59.258 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:59.259 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:59.259 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:59.259 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:59.259 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:59.259 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:59.260 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:51:59.280 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: test_get_num_active_clean: run_osd td/ceph-helpers 0 2026-02-17T04:51:59.280 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:51:59.280 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:51:59.280 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:51:59.280 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:51:59.281 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:51:59.282 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:51:59.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:51:59.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:51:59.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:51:59.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:51:59.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:51:59.283 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:51:59.284 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:51:59.284 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7f9ec5a-6a98-42c0-bf69-976a70d704bf 2026-02-17T04:51:59.284 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 c7f9ec5a-6a98-42c0-bf69-976a70d704bf 2026-02-17T04:51:59.284 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c7f9ec5a-6a98-42c0-bf69-976a70d704bf' 2026-02-17T04:51:59.285 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:51:59.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDv85Np7EJvERAAv48DnTHmBCw8T4JApoPOZQ== 2026-02-17T04:51:59.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDv85Np7EJvERAAv48DnTHmBCw8T4JApoPOZQ=="}' 2026-02-17T04:51:59.294 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7f9ec5a-6a98-42c0-bf69-976a70d704bf -i td/ceph-helpers/0/new.json 2026-02-17T04:51:59.406 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:51:59.414 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:51:59.415 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDv85Np7EJvERAAv48DnTHmBCw8T4JApoPOZQ== --osd-uuid c7f9ec5a-6a98-42c0-bf69-976a70d704bf 2026-02-17T04:51:59.431 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:59.424+0000 7f9898c44a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:59.434 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:59.428+0000 7f9898c44a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:59.435 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:59.428+0000 7f9898c44a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:51:59.435 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:59.428+0000 7f9898c44a40 -1 bdev(0x56041c022400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:51:59.435 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:51:59.428+0000 7f9898c44a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:00.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:00.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:00.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:00.142 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:00.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:00.267 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:00.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:00.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:00.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:00.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:00.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:00.293 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:00.284+0000 7ff6a23a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:00.295 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:00.288+0000 7ff6a23a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:00.296 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:00.288+0000 7ff6a23a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:00.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:00.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:00.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:00.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:00.378 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:00.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:00.379 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:00.379 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:00.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:00.380 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:00.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:00.783 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:00.776+0000 7ff6a23a1a40 -1 Falling back to public interface 2026-02-17T04:52:01.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:01.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:01.494 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:01.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:01.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:01.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:01.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:01.758 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:01.752+0000 7ff6a23a1a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:52:02.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:02.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:02.603 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:02.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:02.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:02.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:02.821 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1172090490,v1:127.0.0.1:6801/1172090490] [v2:127.0.0.1:6802/1172090490,v1:127.0.0.1:6803/1172090490] exists,up c7f9ec5a-6a98-42c0-bf69-976a70d704bf 2026-02-17T04:52:02.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:02.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:02.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:02.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1369: test_get_num_active_clean: create_rbd_pool 2026-02-17T04:52:02.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:52:03.031 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:52:03.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:52:03.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:52:03.321 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:52:03.330 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:52:04.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:52:04.642 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1370: test_get_num_active_clean: wait_for_clean 2026-02-17T04:52:04.642 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:52:04.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:52:04.643 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:52:04.643 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:52:04.643 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:52:04.644 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:52:04.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:52:04.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:52:04.644 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:52:04.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:52:04.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:52:04.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:52:04.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:52:04.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:52:04.746 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:52:04.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:52:04.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:52:04.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:52:04.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:52:05.043 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:52:05.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:52:05.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:52:05.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:52:05.044 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:05.044 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:52:05.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:52:05.045 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:05.046 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:52:05.046 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:52:05.047 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:52:05.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:52:05.048 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:05.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:52:05.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:52:06.270 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:52:06.270 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:06.485 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:52:06.486 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:52:06.486 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:06.486 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:06.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:52:06.781 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:06.995 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:52:06.995 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:52:06.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:06.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:07.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:52:07.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: get_num_active_clean 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:52:07.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:07.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: local num_active_clean=4 2026-02-17T04:52:07.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1372: test_get_num_active_clean: test 4 = 4 2026-02-17T04:52:07.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1373: test_get_num_active_clean: teardown td/ceph-helpers 2026-02-17T04:52:07.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:07.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:07.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:07.604 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:07.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:07.605 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:07.606 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:07.606 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:07.606 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:07.607 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:07.608 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:07.608 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:07.608 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:07.609 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:07.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:07.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:07.616 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:07.616 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.616 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_or_peered td/ceph-helpers 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1393: test_get_num_active_or_peered: local dir=td/ceph-helpers 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1395: test_get_num_active_or_peered: setup td/ceph-helpers 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:52:07.619 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:07.620 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:07.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:07.621 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:07.621 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:07.622 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:07.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:07.623 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:07.623 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:07.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:07.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:07.625 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:07.625 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:07.625 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:07.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:07.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:07.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:07.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.628 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:07.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:07.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:07.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:52:07.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:52:07.631 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.631 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:52:07.632 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1396: test_get_num_active_or_peered: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:52:07.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:52:07.635 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:07.654 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:07.655 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:07.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:52:07.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:52:07.692 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:07.692 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:07.692 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:52:07.693 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:52:07.693 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:07.693 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:07.694 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:52:07.751 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.752 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:07.753 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:07.753 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:52:07.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1397: test_get_num_active_or_peered: run_mgr td/ceph-helpers x 2026-02-17T04:52:07.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:52:07.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:52:07.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:52:07.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:52:07.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:52:07.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:52:07.921 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:52:07.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:07.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:07.922 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:07.922 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.922 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.922 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:07.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:07.923 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:07.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1398: test_get_num_active_or_peered: run_osd td/ceph-helpers 0 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:52:07.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:07.947 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:07.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:07.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:07.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:52:07.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:52:07.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:07.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e62a06aa-6dcf-4da0-9362-06582b7c5ee9 2026-02-17T04:52:07.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e62a06aa-6dcf-4da0-9362-06582b7c5ee9' 2026-02-17T04:52:07.950 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 e62a06aa-6dcf-4da0-9362-06582b7c5ee9 2026-02-17T04:52:07.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:07.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD385Np+EgMORAA99syezbbZaDBPvFo+f29OQ== 2026-02-17T04:52:07.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD385Np+EgMORAA99syezbbZaDBPvFo+f29OQ=="}' 2026-02-17T04:52:07.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e62a06aa-6dcf-4da0-9362-06582b7c5ee9 -i td/ceph-helpers/0/new.json 2026-02-17T04:52:08.069 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:08.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:52:08.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD385Np+EgMORAA99syezbbZaDBPvFo+f29OQ== --osd-uuid e62a06aa-6dcf-4da0-9362-06582b7c5ee9 2026-02-17T04:52:08.094 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.088+0000 7f8f09662a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:08.097 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.092+0000 7f8f09662a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:08.098 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.092+0000 7f8f09662a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:08.098 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.092+0000 7f8f09662a40 -1 bdev(0x5556fa63a400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:08.098 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.092+0000 7f8f09662a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:08.775 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:08.775 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:08.776 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:08.776 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:08.777 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:08.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:08.889 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:08.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:08.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:08.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:08.889 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:08.904 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.896+0000 7fe75399fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:08.905 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.900+0000 7fe75399fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:08.906 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:08.900+0000 7fe75399fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:09.003 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:09.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:09.303 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:09.296+0000 7fe75399fa40 -1 Falling back to public interface 2026-02-17T04:52:09.942 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:09.936+0000 7fe75399fa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:52:10.112 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:10.112 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:10.113 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:10.113 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:10.113 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:10.113 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:10.249 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:11.251 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:11.251 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:11.251 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:11.252 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:11.252 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:11.252 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:11.470 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3003758329,v1:127.0.0.1:6801/3003758329] [v2:127.0.0.1:6802/3003758329,v1:127.0.0.1:6803/3003758329] exists,up e62a06aa-6dcf-4da0-9362-06582b7c5ee9 2026-02-17T04:52:11.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:11.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:11.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:11.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1399: test_get_num_active_or_peered: create_rbd_pool 2026-02-17T04:52:11.470 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:52:11.682 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:52:11.692 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:52:11.692 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:52:11.933 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:52:11.944 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:52:12.945 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:52:13.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1400: test_get_num_active_or_peered: wait_for_clean 2026-02-17T04:52:13.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:52:13.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:52:13.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:52:13.224 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:52:13.225 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:52:13.225 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:52:13.226 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:52:13.226 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:52:13.226 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:52:13.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:52:13.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:52:13.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:52:13.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:52:13.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:52:13.321 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:52:13.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:52:13.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:52:13.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:52:13.529 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:52:13.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:52:13.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:52:13.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:52:13.611 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:52:13.611 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:13.611 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:52:13.611 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:52:13.612 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:13.612 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:52:13.613 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:52:13.613 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:52:13.614 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:52:13.614 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:13.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:52:13.835 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:52:14.835 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:52:14.836 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:15.048 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:52:15.049 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:52:15.049 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:15.049 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:15.326 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:52:15.326 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:52:15.326 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:52:15.327 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:52:15.327 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:52:15.327 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:52:15.327 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:52:15.327 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:15.534 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:52:15.535 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:52:15.535 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:15.536 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:15.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:52:15.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:52:15.826 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:52:15.826 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: get_num_active_or_peered 2026-02-17T04:52:15.826 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-17T04:52:15.826 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-17T04:52:15.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-17T04:52:15.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-17T04:52:15.827 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:16.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: local num_peered=4 2026-02-17T04:52:16.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1402: test_get_num_active_or_peered: test 4 = 4 2026-02-17T04:52:16.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1403: test_get_num_active_or_peered: teardown td/ceph-helpers 2026-02-17T04:52:16.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:16.044 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:16.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:16.045 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:16.045 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:16.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:16.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:16.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:16.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:16.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:16.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:16.154 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:16.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:16.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:16.155 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:16.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:16.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:16.157 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:16.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:16.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:16.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:16.159 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:16.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:16.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:16.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:16.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:16.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:52:16.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_pgs td/ceph-helpers 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1420: test_get_num_pgs: local dir=td/ceph-helpers 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1422: test_get_num_pgs: setup td/ceph-helpers 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:16.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:16.170 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:16.170 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:16.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:16.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:16.170 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:16.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:16.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:16.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:16.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:16.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:16.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:16.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:16.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:16.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:16.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:16.173 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:16.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:16.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:16.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:16.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:16.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.176 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.176 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:16.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:16.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:16.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:52:16.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:52:16.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:52:16.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1423: test_get_num_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:52:16.182 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:52:16.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:52:16.183 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:16.204 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:52:16.204 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:16.204 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:16.205 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:16.205 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.205 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.205 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:16.205 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:16.232 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:52:16.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:52:16.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:16.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:16.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:52:16.233 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:52:16.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:16.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:16.233 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:16.234 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:16.234 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.234 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.234 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:16.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:16.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:16.292 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:16.293 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:16.294 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.294 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.294 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:16.294 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:16.294 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1424: test_get_num_pgs: run_mgr td/ceph-helpers x 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:52:16.351 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:52:16.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:52:16.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:16.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:16.467 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:16.467 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.467 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.467 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:16.467 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:16.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:16.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: test_get_num_pgs: run_osd td/ceph-helpers 0 2026-02-17T04:52:16.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:16.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:16.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:16.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:16.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:52:16.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:52:16.493 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:16.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1f97e2e3-ffad-49b3-80e6-85e4c6b18b99 2026-02-17T04:52:16.493 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 1f97e2e3-ffad-49b3-80e6-85e4c6b18b99 2026-02-17T04:52:16.494 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1f97e2e3-ffad-49b3-80e6-85e4c6b18b99' 2026-02-17T04:52:16.494 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:16.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAA9JNpExPcHRAAbjcplfa0VgWs6T9GpYth4Q== 2026-02-17T04:52:16.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAA9JNpExPcHRAAbjcplfa0VgWs6T9GpYth4Q=="}' 2026-02-17T04:52:16.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1f97e2e3-ffad-49b3-80e6-85e4c6b18b99 -i td/ceph-helpers/0/new.json 2026-02-17T04:52:16.614 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:16.622 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:52:16.623 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAA9JNpExPcHRAAbjcplfa0VgWs6T9GpYth4Q== --osd-uuid 1f97e2e3-ffad-49b3-80e6-85e4c6b18b99 2026-02-17T04:52:16.639 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:16.636+0000 7fae43f10a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:16.641 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:16.636+0000 7fae43f10a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:16.641 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:16.640+0000 7fae43f10a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:16.642 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:16.640+0000 7fae43f10a40 -1 bdev(0x561626cd2400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:16.643 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:16.640+0000 7fae43f10a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:17.397 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:17.397 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:17.398 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:17.399 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:17.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:17.522 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:17.523 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:17.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:17.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:17.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:17.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:17.547 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:17.544+0000 7fd0a2349a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:17.549 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:17.548+0000 7fd0a2349a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:17.550 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:17.548+0000 7fd0a2349a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:17.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:17.745 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:18.095 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:18.092+0000 7fd0a2349a40 -1 Falling back to public interface 2026-02-17T04:52:18.745 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:18.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:18.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:18.746 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:18.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:18.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:18.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:18.964 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:18.960+0000 7fd0a2349a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:52:19.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:19.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:19.856 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:19.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:19.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:19.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:20.073 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2186156528,v1:127.0.0.1:6801/2186156528] [v2:127.0.0.1:6802/2186156528,v1:127.0.0.1:6803/2186156528] exists,up 1f97e2e3-ffad-49b3-80e6-85e4c6b18b99 2026-02-17T04:52:20.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:20.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:20.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:20.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1426: test_get_num_pgs: create_rbd_pool 2026-02-17T04:52:20.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:52:20.289 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:52:20.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:52:20.299 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:52:20.517 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:52:20.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:52:21.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:52:21.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1427: test_get_num_pgs: wait_for_clean 2026-02-17T04:52:21.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:52:21.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:52:21.852 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:52:21.852 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:52:21.853 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:52:21.853 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:52:21.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:52:21.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:52:21.854 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:52:21.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:52:21.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:52:21.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:52:21.955 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:52:21.955 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:52:21.955 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:52:22.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:52:22.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:52:22.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:52:22.173 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:52:22.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:52:22.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:52:22.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:52:22.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:52:22.253 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:22.254 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:52:22.255 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:52:22.255 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:22.255 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:52:22.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:52:22.256 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:52:22.256 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:52:22.257 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:22.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:52:22.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:52:23.482 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:52:23.483 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:23.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:52:23.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:52:23.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:23.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:52:23.983 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:24.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:52:24.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:52:24.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:24.188 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:24.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:52:24.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:52:24.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:52:24.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: get_num_pgs 2026-02-17T04:52:24.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:24.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:24.748 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: local num_pgs=4 2026-02-17T04:52:24.748 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1429: test_get_num_pgs: test 4 -gt 0 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1430: test_get_num_pgs: teardown td/ceph-helpers 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:24.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:24.856 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:24.857 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:24.857 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:24.858 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:24.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:24.859 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:24.860 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:24.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:24.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:24.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:24.861 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:24.862 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:24.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:24.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:24.867 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:24.867 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:24.867 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:24.867 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_op_scheduler td/ceph-helpers 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: test_get_op_scheduler: local dir=td/ceph-helpers 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: test_get_op_scheduler: setup td/ceph-helpers 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:52:24.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:24.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:24.872 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:24.872 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:24.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:24.873 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:24.874 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:24.874 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:24.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:24.875 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:24.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:24.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:24.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:24.877 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:24.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:24.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:24.879 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:24.879 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:24.879 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:24.879 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:24.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:24.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:24.880 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:52:24.881 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:52:24.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:24.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:24.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:52:24.883 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:52:24.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:52:24.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:52:24.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:52:24.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: test_get_op_scheduler: run_mon td/ceph-helpers a 2026-02-17T04:52:24.884 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:52:24.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:52:24.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:52:24.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:52:24.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:52:24.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T04:52:24.906 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:52:24.906 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:24.906 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:24.907 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:24.907 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:24.907 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:24.907 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:24.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T04:52:24.939 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:52:24.939 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:52:24.940 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:24.940 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:24.940 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:52:24.940 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:52:24.941 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:24.941 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:24.941 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:24.941 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:24.942 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:24.942 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:24.942 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:24.942 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:24.942 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:52:24.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:52:24.996 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:24.997 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:24.998 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:24.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:24.998 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:52:25.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: test_get_op_scheduler: run_mgr td/ceph-helpers x 2026-02-17T04:52:25.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:52:25.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:52:25.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:52:25.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:52:25.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:52:25.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:52:25.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:52:25.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:25.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:25.163 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:25.163 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:25.163 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:25.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:25.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:25.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:25.185 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: test_get_op_scheduler: run_osd td/ceph-helpers 0 --osd_op_queue=wpq 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:25.186 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:25.187 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=wpq 2026-02-17T04:52:25.188 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:52:25.189 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:25.189 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=24b76ef0-a494-40d6-a653-2bb1351ec60b 2026-02-17T04:52:25.189 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 24b76ef0-a494-40d6-a653-2bb1351ec60b' 2026-02-17T04:52:25.190 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 24b76ef0-a494-40d6-a653-2bb1351ec60b 2026-02-17T04:52:25.190 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:25.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAJ9JNp3/zBCxAAiylQCAhokiMCFZzse7GC3A== 2026-02-17T04:52:25.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAJ9JNp3/zBCxAAiylQCAhokiMCFZzse7GC3A=="}' 2026-02-17T04:52:25.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 24b76ef0-a494-40d6-a653-2bb1351ec60b -i td/ceph-helpers/0/new.json 2026-02-17T04:52:25.311 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:25.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:52:25.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq --mkfs --key AQAJ9JNp3/zBCxAAiylQCAhokiMCFZzse7GC3A== --osd-uuid 24b76ef0-a494-40d6-a653-2bb1351ec60b 2026-02-17T04:52:25.336 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:25.332+0000 7fab70edaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:25.338 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:25.336+0000 7fab70edaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:25.339 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:25.336+0000 7fab70edaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:25.340 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:25.336+0000 7fab70edaa40 -1 bdev(0x5617de0c8400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:25.340 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:25.336+0000 7fab70edaa40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:26.098 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:26.098 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:26.099 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:26.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:26.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:26.222 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:26.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:26.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq 2026-02-17T04:52:26.223 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:26.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:26.224 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:26.247 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:26.244+0000 7f9ef2cc5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:26.249 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:26.248+0000 7f9ef2cc5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:26.251 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:26.248+0000 7f9ef2cc5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:26.330 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:26.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:26.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:26.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:26.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:26.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:26.331 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:26.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:26.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:26.332 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:26.437 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:26.711 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:26.708+0000 7f9ef2cc5a40 -1 Falling back to public interface 2026-02-17T04:52:27.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:27.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:27.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:27.439 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:27.440 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:27.440 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:27.546 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:27.703 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:27.700+0000 7f9ef2cc5a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:52:28.548 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:28.548 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:28.548 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:28.549 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:28.549 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:28.549 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:28.762 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3790399597,v1:127.0.0.1:6801/3790399597] [v2:127.0.0.1:6802/3790399597,v1:127.0.0.1:6803/3790399597] exists,up 24b76ef0-a494-40d6-a653-2bb1351ec60b 2026-02-17T04:52:28.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:28.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: get_op_scheduler 0 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=0 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 0 osd_op_queue 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T04:52:28.764 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T04:52:28.765 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:28.766 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:28.766 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:28.766 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.0.asok 2026-02-17T04:52:28.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:28.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-osd.0.asok config get osd_op_queue 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: test wpq = wpq 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2325: test_get_op_scheduler: run_osd td/ceph-helpers 1 --osd_op_queue=mclock_scheduler 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-17T04:52:28.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:28.830 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:28.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:28.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:28.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:28.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:28.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:28.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T04:52:28.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-17T04:52:28.833 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:28.833 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ba8858e4-d601-476a-adfb-26c169ec2284 2026-02-17T04:52:28.833 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 ba8858e4-d601-476a-adfb-26c169ec2284 2026-02-17T04:52:28.834 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ba8858e4-d601-476a-adfb-26c169ec2284' 2026-02-17T04:52:28.834 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:28.843 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAM9JNpcx8hMhAAVVXj+FopFrs1o2VR6QkoFA== 2026-02-17T04:52:28.843 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAM9JNpcx8hMhAAVVXj+FopFrs1o2VR6QkoFA=="}' 2026-02-17T04:52:28.843 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ba8858e4-d601-476a-adfb-26c169ec2284 -i td/ceph-helpers/1/new.json 2026-02-17T04:52:29.075 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:29.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-17T04:52:29.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQAM9JNpcx8hMhAAVVXj+FopFrs1o2VR6QkoFA== --osd-uuid ba8858e4-d601-476a-adfb-26c169ec2284 2026-02-17T04:52:29.112 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:29.107+0000 7fa04d0f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:29.114 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:29.111+0000 7fa04d0f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:29.115 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:29.111+0000 7fa04d0f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:29.115 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:29.111+0000 7fa04d0f2a40 -1 bdev(0x56385c796400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:29.115 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:29.111+0000 7fa04d0f2a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-17T04:52:29.781 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-17T04:52:29.781 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:29.782 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:52:29.782 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:52:29.782 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:30.086 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:52:30.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:52:30.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T04:52:30.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:30.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:30.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:30.103 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:30.099+0000 7fb750e08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:30.104 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:30.099+0000 7fb750e08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:30.105 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:30.103+0000 7fb750e08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:30.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:52:30.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:30.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:52:30.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:30.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:30.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:30.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:30.303 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:30.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:30.304 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:52:30.515 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:30.555 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:30.551+0000 7fb750e08a40 -1 Falling back to public interface 2026-02-17T04:52:31.127 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:31.123+0000 7fb750e08a40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:52:31.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:31.517 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:31.517 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:31.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:31.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:31.518 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:52:31.732 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:32.733 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:32.733 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:32.734 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:32.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:32.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:32.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:52:32.953 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/373400587,v1:127.0.0.1:6811/373400587] [v2:127.0.0.1:6812/373400587,v1:127.0.0.1:6813/373400587] exists,up ba8858e4-d601-476a-adfb-26c169ec2284 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: get_op_scheduler 1 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=1 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 1 osd_op_queue 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=1 2026-02-17T04:52:32.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.1 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:32.955 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:32.956 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-osd.1.asok 2026-02-17T04:52:32.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:32.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-osd.1.asok config get osd_op_queue 2026-02-17T04:52:33.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: test mclock_scheduler = mclock_scheduler 2026-02-17T04:52:33.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: teardown td/ceph-helpers 2026-02-17T04:52:33.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:33.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:33.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:33.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:33.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:33.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:33.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:33.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:33.126 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:33.127 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:33.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:33.128 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:33.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:33.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:33.130 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:33.131 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:33.131 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:33.131 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:33.132 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:33.132 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:33.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:33.134 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:33.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:33.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.138 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:33.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osd_id_used_by_pgs td/ceph-helpers 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1450: test_get_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1452: test_get_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:33.141 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:33.142 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:33.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:33.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:33.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:33.142 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:33.143 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:33.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:33.144 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:33.145 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:33.145 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:33.146 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:33.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:33.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:33.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:33.147 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:33.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:33.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:33.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:33.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:33.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:33.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:33.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:33.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:52:33.153 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:52:33.153 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.153 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:52:33.154 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:52:33.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:52:33.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1453: test_get_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:52:33.156 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:33.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:52:33.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:33.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:33.178 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:33.178 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.178 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:33.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:33.209 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:52:33.210 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:52:33.211 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:33.211 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:33.211 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:52:33.211 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:52:33.212 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:33.212 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:33.212 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:33.212 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:33.212 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.213 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.213 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:33.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:33.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:52:33.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:52:33.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:33.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:33.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:52:33.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:52:33.270 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:33.270 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:33.271 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:52:33.326 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1454: test_get_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-17T04:52:33.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:52:33.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:52:33.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:52:33.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:52:33.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:52:33.327 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.441 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:33.442 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:33.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:33.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1455: test_get_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-17T04:52:33.464 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:52:33.465 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:33.466 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:52:33.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:52:33.468 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:33.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9de40543-6864-404f-b82f-03562503e508 2026-02-17T04:52:33.468 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 9de40543-6864-404f-b82f-03562503e508 2026-02-17T04:52:33.469 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9de40543-6864-404f-b82f-03562503e508' 2026-02-17T04:52:33.469 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:33.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAR9JNpFQxhHBAAN7YyaqA3M7zhj4frhDOk5Q== 2026-02-17T04:52:33.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAR9JNpFQxhHBAAN7YyaqA3M7zhj4frhDOk5Q=="}' 2026-02-17T04:52:33.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9de40543-6864-404f-b82f-03562503e508 -i td/ceph-helpers/0/new.json 2026-02-17T04:52:33.587 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:33.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:52:33.596 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAR9JNpFQxhHBAAN7YyaqA3M7zhj4frhDOk5Q== --osd-uuid 9de40543-6864-404f-b82f-03562503e508 2026-02-17T04:52:33.612 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:33.607+0000 7fb227435a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:33.613 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:33.611+0000 7fb227435a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:33.614 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:33.611+0000 7fb227435a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:33.614 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:33.611+0000 7fb227435a40 -1 bdev(0x5620e1462400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:33.615 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:33.611+0000 7fb227435a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:34.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:34.305 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:34.306 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:34.306 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:34.307 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:34.418 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:34.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:34.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:34.419 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:34.420 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:34.420 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:34.434 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:34.431+0000 7f1e9a14aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:34.435 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:34.431+0000 7f1e9a14aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:34.437 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:34.431+0000 7f1e9a14aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:34.529 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:34.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:34.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:34.530 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:34.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:34.979 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:34.975+0000 7f1e9a14aa40 -1 Falling back to public interface 2026-02-17T04:52:35.639 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:35.639 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:35.639 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:35.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:35.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:35.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:35.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:35.990 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:35.987+0000 7f1e9a14aa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:52:36.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:36.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:36.797 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:36.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:36.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:36.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:37.018 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3970387141,v1:127.0.0.1:6801/3970387141] [v2:127.0.0.1:6802/3970387141,v1:127.0.0.1:6803/3970387141] exists,up 9de40543-6864-404f-b82f-03562503e508 2026-02-17T04:52:37.018 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:37.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:37.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:37.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1456: test_get_osd_id_used_by_pgs: create_rbd_pool 2026-02-17T04:52:37.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:52:37.231 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:52:37.241 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:52:37.242 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:52:37.462 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:52:37.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:52:38.473 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:52:38.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1457: test_get_osd_id_used_by_pgs: wait_for_clean 2026-02-17T04:52:38.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:52:38.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:52:38.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:52:38.753 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:52:38.754 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:52:38.754 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:52:38.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:52:38.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:52:38.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:52:38.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:52:38.850 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:52:38.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:52:38.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:52:38.851 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:52:38.851 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:52:39.062 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:52:39.063 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:52:39.063 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:52:39.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:52:39.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:52:39.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:52:39.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:52:39.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:52:39.144 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:39.144 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:52:39.145 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:52:39.146 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:52:39.146 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:52:39.147 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:52:39.147 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:52:39.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:52:39.148 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:39.370 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:52:39.371 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:52:40.371 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:52:40.372 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:40.589 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:52:40.590 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:52:40.591 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:40.591 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:52:40.882 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:41.093 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:52:41.094 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:52:41.094 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:41.094 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:41.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:52:41.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:52:41.376 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:52:41.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-17T04:52:41.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: uniq 2026-02-17T04:52:41.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-17T04:52:41.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-17T04:52:41.377 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-17T04:52:41.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: local osd_ids=0 2026-02-17T04:52:41.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1459: test_get_osd_id_used_by_pgs: test 0 = 0 2026-02-17T04:52:41.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1460: test_get_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-17T04:52:41.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:41.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:41.694 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:41.694 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:41.694 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:41.695 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:41.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:41.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:41.696 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:41.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:41.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:41.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:41.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:41.698 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:41.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:41.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:41.705 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:41.705 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:41.705 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:41.705 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osds td/ceph-helpers 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: test_get_osds: local dir=td/ceph-helpers 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1024: test_get_osds: setup td/ceph-helpers 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:41.707 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:41.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:41.708 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:41.709 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:41.710 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:41.710 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:41.711 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:41.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:41.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:41.712 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:41.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:41.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:41.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:41.714 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:41.715 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:41.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:41.716 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:41.717 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:41.717 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:41.718 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:41.718 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:41.719 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:41.719 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:41.719 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:52:41.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:52:41.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:41.721 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:41.721 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:52:41.722 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: test_get_osds: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:52:41.723 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-17T04:52:41.745 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:52:41.745 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:41.746 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:41.746 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:41.746 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:41.746 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:41.746 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:41.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T04:52:41.783 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:52:41.784 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:52:41.784 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:41.784 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:41.784 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:52:41.784 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:41.785 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:41.786 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:41.786 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:52:41.842 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:52:41.842 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:41.842 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:41.842 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:52:41.842 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:41.843 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:41.844 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:41.844 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1026: test_get_osds: run_mgr td/ceph-helpers x 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:52:41.899 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:52:42.015 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:52:42.015 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:42.015 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:42.016 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:42.016 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:42.016 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:42.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:42.016 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:42.017 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:42.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: test_get_osds: run_osd td/ceph-helpers 0 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:42.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:42.040 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:42.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:42.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:42.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:42.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:52:42.042 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:52:42.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:42.043 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dbdb8407-48e0-43aa-89be-7585fc06e991 2026-02-17T04:52:42.043 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dbdb8407-48e0-43aa-89be-7585fc06e991' 2026-02-17T04:52:42.043 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 dbdb8407-48e0-43aa-89be-7585fc06e991 2026-02-17T04:52:42.043 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:42.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAa9JNpg3HtAhAAB0eX3FD9JRwk1Fm6WE4CDQ== 2026-02-17T04:52:42.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAa9JNpg3HtAhAAB0eX3FD9JRwk1Fm6WE4CDQ=="}' 2026-02-17T04:52:42.051 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dbdb8407-48e0-43aa-89be-7585fc06e991 -i td/ceph-helpers/0/new.json 2026-02-17T04:52:42.164 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:42.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:52:42.173 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAa9JNpg3HtAhAAB0eX3FD9JRwk1Fm6WE4CDQ== --osd-uuid dbdb8407-48e0-43aa-89be-7585fc06e991 2026-02-17T04:52:42.188 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:42.183+0000 7ff81b5bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:42.191 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:42.187+0000 7ff81b5bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:42.192 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:42.187+0000 7ff81b5bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:42.192 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:42.187+0000 7ff81b5bea40 -1 bdev(0x55ecc2250400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:42.193 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:42.187+0000 7ff81b5bea40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:42.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:42.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:42.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:42.886 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:42.886 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:42.999 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:42.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:42.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:42.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:42.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:42.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:43.014 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:43.011+0000 7f256045fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:43.016 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:43.011+0000 7f256045fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:43.017 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:43.015+0000 7f256045fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:43.114 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:43.115 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:43.225 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:43.447 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:43.443+0000 7f256045fa40 -1 Falling back to public interface 2026-02-17T04:52:44.115 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:44.111+0000 7f256045fa40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:52:44.226 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:44.227 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:44.227 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:44.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:44.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:44.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:44.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:45.343 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:45.344 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:45.344 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:45.344 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:45.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:45.345 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:45.564 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1761629309,v1:127.0.0.1:6801/1761629309] [v2:127.0.0.1:6802/1761629309,v1:127.0.0.1:6803/1761629309] exists,up dbdb8407-48e0-43aa-89be-7585fc06e991 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1028: test_get_osds: run_osd td/ceph-helpers 1 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:45.568 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:45.569 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:45.570 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-17T04:52:45.571 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:45.572 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f5c06f47-452c-4bfd-b7c6-0e45e0691482 2026-02-17T04:52:45.572 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f5c06f47-452c-4bfd-b7c6-0e45e0691482' 2026-02-17T04:52:45.573 INFO:tasks.workunit.client.0.trial105.stdout:add osd1 f5c06f47-452c-4bfd-b7c6-0e45e0691482 2026-02-17T04:52:45.573 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:45.581 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAd9JNpb9yEIhAAxvFw5hB5oxlhhsJpPlLN9w== 2026-02-17T04:52:45.581 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAd9JNpb9yEIhAAxvFw5hB5oxlhhsJpPlLN9w=="}' 2026-02-17T04:52:45.581 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f5c06f47-452c-4bfd-b7c6-0e45e0691482 -i td/ceph-helpers/1/new.json 2026-02-17T04:52:45.806 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:45.815 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-17T04:52:45.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAd9JNpb9yEIhAAxvFw5hB5oxlhhsJpPlLN9w== --osd-uuid f5c06f47-452c-4bfd-b7c6-0e45e0691482 2026-02-17T04:52:45.832 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:45.827+0000 7f3307334a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:45.834 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:45.831+0000 7f3307334a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:45.835 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:45.831+0000 7f3307334a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:45.835 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:45.831+0000 7f3307334a40 -1 bdev(0x562d56f48400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:45.835 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:45.831+0000 7f3307334a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-17T04:52:46.484 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-17T04:52:46.484 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:46.485 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T04:52:46.485 INFO:tasks.workunit.client.0.trial105.stdout:adding osd1 key to auth repository 2026-02-17T04:52:46.486 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:46.781 INFO:tasks.workunit.client.0.trial105.stdout:start osd.1 2026-02-17T04:52:46.782 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T04:52:46.782 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:46.783 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:46.783 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:46.783 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:46.806 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:46.803+0000 7f8272676a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:46.808 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:46.803+0000 7f8272676a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:46.809 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:46.807+0000 7f8272676a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:46.992 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T04:52:46.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:46.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T04:52:46.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:46.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:46.993 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:46.993 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:46.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:46.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:46.994 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:52:47.175 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:47.171+0000 7f8272676a40 -1 Falling back to public interface 2026-02-17T04:52:47.210 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:47.812 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:47.807+0000 7f8272676a40 -1 osd.1 0 log_to_monitors true 2026-02-17T04:52:48.211 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:48.211 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:48.212 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:48.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:48.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:48.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:52:48.432 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:49.433 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:49.433 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:49.434 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:52:49.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:52:49.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:49.434 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T04:52:49.649 INFO:tasks.workunit.client.0.trial105.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4271146933,v1:127.0.0.1:6811/4271146933] [v2:127.0.0.1:6812/4271146933,v1:127.0.0.1:6813/4271146933] exists,up f5c06f47-452c-4bfd-b7c6-0e45e0691482 2026-02-17T04:52:49.650 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:52:49.650 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:52:49.650 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:52:49.651 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1029: test_get_osds: create_rbd_pool 2026-02-17T04:52:49.651 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:52:49.859 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:52:49.869 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:52:49.869 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:52:50.152 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:52:50.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:52:51.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:52:51.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1030: test_get_osds: wait_for_clean 2026-02-17T04:52:51.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:52:51.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:52:51.399 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:52:51.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:52:51.400 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:52:51.400 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:52:51.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:52:51.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:52:51.400 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:52:51.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:52:51.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:52:51.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:52:51.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:52:51.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:52:51.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:52:51.712 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T04:52:51.713 INFO:tasks.workunit.client.0.trial105.stderr:1' 2026-02-17T04:52:51.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:52:51.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:52:51.713 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:52:51.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-17T04:52:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-17T04:52:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T04:52:51.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:52:51.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T04:52:51.875 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-17T04:52:51.875 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-17T04:52:51.875 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-17T04:52:51.875 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:52:51.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-17T04:52:51.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:52:51.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:52:51.877 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-17T04:52:51.877 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:52:51.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-17T04:52:51.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T04:52:51.879 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836484 2026-02-17T04:52:51.879 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:52.095 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-17T04:52:52.095 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:52:53.096 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:52:53.097 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:52:53.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T04:52:53.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:52:53.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672963 2026-02-17T04:52:53.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:52:53.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T04:52:53.318 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-17T04:52:53.318 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:52:53.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672963 2026-02-17T04:52:53.320 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.1 seq 42949672963 2026-02-17T04:52:53.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-17T04:52:53.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T04:52:53.543 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-17T04:52:53.544 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:52:53.544 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:53.544 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:53.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:52:53.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:52:54.036 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:52:54.037 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:52:54.037 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:52:54.037 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:52:54.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:52:54.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:52:54.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:52:54.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: test_get_osds: create_rbd_pool 2026-02-17T04:52:54.331 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:52:54.592 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:52:54.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:52:54.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:52:54.859 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:52:54.868 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:52:55.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:52:56.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: get_osds rbd GROUP 2026-02-17T04:52:56.151 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=rbd 2026-02-17T04:52:56.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: grep --quiet '^[0-1] [0-1]$' 2026-02-17T04:52:56.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=GROUP 2026-02-17T04:52:56.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map rbd GROUP 2026-02-17T04:52:56.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-17T04:52:56.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-17T04:52:56.362 INFO:tasks.workunit.client.0.trial105.stderr:0' 2026-02-17T04:52:56.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-17T04:52:56.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1033: test_get_osds: teardown td/ceph-helpers 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:56.363 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:56.471 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:56.471 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:56.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:56.472 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:56.474 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:56.474 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:56.474 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:56.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:56.475 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:56.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:56.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:56.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:56.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:56.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:56.486 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:56.486 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.486 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.486 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:56.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:56.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:56.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:52:56.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_pg td/ceph-helpers 2026-02-17T04:52:56.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1091: test_get_pg: local dir=td/ceph-helpers 2026-02-17T04:52:56.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: test_get_pg: setup td/ceph-helpers 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:52:56.491 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:52:56.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:52:56.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:52:56.492 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:52:56.493 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:52:56.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:52:56.493 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:52:56.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:52:56.493 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:52:56.494 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:52:56.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:56.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:52:56.495 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:52:56.495 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:52:56.496 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:52:56.497 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:52:56.498 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:52:56.499 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:52:56.499 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.499 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.499 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:52:56.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:52:56.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:52:56.500 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:52:56.501 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:52:56.501 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.502 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:52:56.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:52:56.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:52:56.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:52:56.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: test_get_pg: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:52:56.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:56.527 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:52:56.527 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:56.527 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:56.527 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:56.528 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.528 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:56.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:52:56.555 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:52:56.556 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:52:56.556 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:56.557 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:56.557 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:52:56.558 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:52:56.558 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:56.558 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:56.558 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:56.558 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:56.558 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.559 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.559 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:56.559 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:56.559 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:52:56.627 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:52:56.628 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:52:56.629 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:52:56.629 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.630 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.630 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:52:56.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:52:56.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1095: test_get_pg: run_mgr td/ceph-helpers x 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:52:56.683 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.793 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:56.794 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:56.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:52:56.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: test_get_pg: run_osd td/ceph-helpers 0 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:52:56.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:52:56.818 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:52:56.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:52:56.820 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:52:56.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ec5af303-5eb5-47e4-89ba-3850e6814223 2026-02-17T04:52:56.820 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ec5af303-5eb5-47e4-89ba-3850e6814223' 2026-02-17T04:52:56.820 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 ec5af303-5eb5-47e4-89ba-3850e6814223 2026-02-17T04:52:56.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:52:56.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAo9JNpetNjMRAANF5WAl3yKOiQlZFI69aPzA== 2026-02-17T04:52:56.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAo9JNpetNjMRAANF5WAl3yKOiQlZFI69aPzA=="}' 2026-02-17T04:52:56.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ec5af303-5eb5-47e4-89ba-3850e6814223 -i td/ceph-helpers/0/new.json 2026-02-17T04:52:56.942 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:56.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:52:56.951 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAo9JNpetNjMRAANF5WAl3yKOiQlZFI69aPzA== --osd-uuid ec5af303-5eb5-47e4-89ba-3850e6814223 2026-02-17T04:52:56.968 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:56.963+0000 7fb2b5c57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:56.969 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:56.967+0000 7fb2b5c57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:56.970 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:56.967+0000 7fb2b5c57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:56.971 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:56.967+0000 7fb2b5c57a40 -1 bdev(0x56351c724400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:52:56.971 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:56.967+0000 7fb2b5c57a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:52:57.741 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:52:57.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:52:57.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:52:57.743 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:52:57.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:52:57.863 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:52:57.863 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:52:57.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:52:57.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:52:57.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:52:57.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:52:57.887 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:57.883+0000 7fc619dc4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:57.890 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:57.887+0000 7fc619dc4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:57.891 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:57.887+0000 7fc619dc4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:52:57.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:52:57.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:52:57.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:52:57.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:52:57.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:52:57.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:57.971 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:52:57.972 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:52:57.972 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:57.972 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:58.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:58.319 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:58.315+0000 7fc619dc4a40 -1 Falling back to public interface 2026-02-17T04:52:59.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:52:59.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:52:59.088 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:52:59.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:52:59.089 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:52:59.089 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:52:59.192 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:52:59.343 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:52:59.339+0000 7fc619dc4a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:53:00.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:00.194 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:00.194 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:53:00.195 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:53:00.195 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:00.195 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:00.406 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3801011645,v1:127.0.0.1:6801/3801011645] [v2:127.0.0.1:6802/3801011645,v1:127.0.0.1:6803/3801011645] exists,up ec5af303-5eb5-47e4-89ba-3850e6814223 2026-02-17T04:53:00.406 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:00.406 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:00.407 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:00.407 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1097: test_get_pg: create_rbd_pool 2026-02-17T04:53:00.407 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:53:00.606 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:53:00.615 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:53:00.616 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:53:00.851 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:53:00.860 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:53:01.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:53:02.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1098: test_get_pg: wait_for_clean 2026-02-17T04:53:02.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:53:02.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:53:02.179 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:53:02.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:53:02.180 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:02.180 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:02.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:02.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:02.180 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:02.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:53:02.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:53:02.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:53:02.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:53:02.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:53:02.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:53:02.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:53:02.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:53:02.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:53:02.503 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:53:02.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:53:02.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:53:02.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:53:02.585 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:53:02.586 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:02.586 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:53:02.586 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:53:02.587 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:02.587 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:53:02.588 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:53:02.588 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:53:02.589 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:53:02.589 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:02.803 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:53:02.804 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:53:03.805 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:53:03.805 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:04.023 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:53:04.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:53:04.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:04.024 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:53:04.320 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:53:04.321 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:53:04.531 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:53:04.531 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:53:04.531 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:04.531 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:04.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:53:04.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:53:04.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:53:04.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: get_pg rbd GROUP 2026-02-17T04:53:04.816 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-17T04:53:04.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: grep --quiet '^[0-9]\.[0-9a-f][0-9a-f]*$' 2026-02-17T04:53:04.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=GROUP 2026-02-17T04:53:04.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd GROUP 2026-02-17T04:53:04.817 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: test_get_pg: teardown td/ceph-helpers 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:05.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:05.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:05.148 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:05.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:05.149 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:05.149 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:05.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:05.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:05.151 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:05.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:05.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:05.152 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:05.152 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:05.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:05.154 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:05.155 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:05.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:05.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.158 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.158 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_primary td/ceph-helpers 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: test_get_primary: local dir=td/ceph-helpers 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: test_get_primary: setup td/ceph-helpers 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:05.161 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:05.162 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:05.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:05.162 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:05.162 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:05.162 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:05.162 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:05.163 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:05.163 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:05.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:05.164 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:05.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:05.165 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:05.166 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:05.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:05.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:05.167 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:05.168 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:05.168 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:05.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:05.169 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:05.171 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:05.171 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.171 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:05.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:05.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:05.172 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:53:05.174 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:53:05.174 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.174 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:53:05.175 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: test_get_primary: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:53:05.177 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:53:05.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:05.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:53:05.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:05.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:05.198 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:05.198 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.198 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.199 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:05.199 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:05.228 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:53:05.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:53:05.229 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:05.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:05.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:53:05.230 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:53:05.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:05.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:05.230 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:05.231 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:05.231 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.231 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.231 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:05.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:05.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:53:05.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:53:05.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:05.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:05.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:53:05.286 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:53:05.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:05.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:05.287 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:05.288 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:05.288 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.288 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.288 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:05.288 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:05.288 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: test_get_primary: local osd=0 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: test_get_primary: run_mgr td/ceph-helpers x 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:53:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:53:05.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:53:05.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:05.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:05.452 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:05.452 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.453 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:05.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:05.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:05.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1201: test_get_primary: run_osd td/ceph-helpers 0 2026-02-17T04:53:05.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:53:05.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:53:05.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:53:05.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:53:05.477 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:53:05.478 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:53:05.479 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:53:05.480 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:53:05.481 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8802d60f-5118-4a93-95b7-a785467a90d5 2026-02-17T04:53:05.481 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 8802d60f-5118-4a93-95b7-a785467a90d5 2026-02-17T04:53:05.482 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8802d60f-5118-4a93-95b7-a785467a90d5' 2026-02-17T04:53:05.482 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:53:05.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAx9JNpLav+HBAAIEbZIle8rXJe+gLBLGKsmg== 2026-02-17T04:53:05.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAx9JNpLav+HBAAIEbZIle8rXJe+gLBLGKsmg=="}' 2026-02-17T04:53:05.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8802d60f-5118-4a93-95b7-a785467a90d5 -i td/ceph-helpers/0/new.json 2026-02-17T04:53:05.603 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:05.611 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:53:05.613 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAx9JNpLav+HBAAIEbZIle8rXJe+gLBLGKsmg== --osd-uuid 8802d60f-5118-4a93-95b7-a785467a90d5 2026-02-17T04:53:05.629 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:05.622+0000 7f7524900a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:05.630 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:05.626+0000 7f7524900a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:05.631 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:05.626+0000 7f7524900a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:05.631 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:05.626+0000 7f7524900a40 -1 bdev(0x56524b614400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:53:05.631 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:05.626+0000 7f7524900a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:53:06.361 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:53:06.361 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:53:06.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:53:06.362 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:53:06.362 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:53:06.486 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:53:06.487 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:53:06.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:53:06.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:53:06.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:53:06.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:53:06.510 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:06.506+0000 7f6fcf3aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:06.513 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:06.506+0000 7f6fcf3aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:06.514 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:06.510+0000 7f6fcf3aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:06.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:53:06.598 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:06.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:06.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:06.711 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:06.871 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:06.866+0000 7f6fcf3aba40 -1 Falling back to public interface 2026-02-17T04:53:07.550 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:07.546+0000 7f6fcf3aba40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:53:07.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:07.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:07.713 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:53:07.713 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:53:07.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:07.714 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:07.822 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:08.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:08.824 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:08.825 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:53:08.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:53:08.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:08.825 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:09.037 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2811945931,v1:127.0.0.1:6801/2811945931] [v2:127.0.0.1:6802/2811945931,v1:127.0.0.1:6803/2811945931] exists,up 8802d60f-5118-4a93-95b7-a785467a90d5 2026-02-17T04:53:09.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:09.038 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:09.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:09.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: test_get_primary: create_rbd_pool 2026-02-17T04:53:09.039 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:53:09.242 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:53:09.250 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:53:09.250 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:53:09.497 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:53:09.508 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:53:10.509 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: test_get_primary: wait_for_clean 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:10.836 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:10.837 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:10.837 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:10.837 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:10.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:53:10.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:53:10.928 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:53:10.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:53:10.929 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:53:10.929 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:53:11.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:53:11.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:53:11.153 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:53:11.154 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:53:11.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:53:11.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:53:11.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:53:11.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:53:11.235 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:11.236 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:53:11.236 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:53:11.237 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:11.237 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:53:11.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:53:11.238 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:53:11.239 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:53:11.239 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:11.460 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:53:11.460 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:53:12.460 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:53:12.461 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:12.670 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:53:12.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:53:12.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:12.671 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:53:12.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:53:13.164 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:53:13.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:53:13.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:13.165 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:13.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:53:13.452 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:53:13.453 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:53:13.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: get_primary rbd GROUP 2026-02-17T04:53:13.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-17T04:53:13.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-17T04:53:13.453 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-17T04:53:13.454 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-17T04:53:13.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: test 0 = 0 2026-02-17T04:53:13.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1205: test_get_primary: teardown td/ceph-helpers 2026-02-17T04:53:13.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:13.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:13.672 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:13.673 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:13.673 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:13.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:13.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:13.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:13.782 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:13.782 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:13.783 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:13.783 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:13.786 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:13.786 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:13.786 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:13.787 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:13.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:13.788 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:13.788 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:13.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:13.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:13.789 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:13.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:13.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:13.795 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:13.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:13.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:13.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:13.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:13.796 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_timeout_delays td/ceph-helpers 2026-02-17T04:53:13.797 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: get_timeout_delays 1 2026-02-17T04:53:13.797 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.797 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.797 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.797 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.797 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: test '1 ' = '1 ' 2026-02-17T04:53:13.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: get_timeout_delays 5 2026-02-17T04:53:13.812 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.812 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: test '1 2 2 ' = '1 2 2 ' 2026-02-17T04:53:13.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: get_timeout_delays 6 2026-02-17T04:53:13.832 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.832 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.832 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.832 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.832 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.855 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: test '1 2 3 ' = '1 2 3 ' 2026-02-17T04:53:13.855 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: get_timeout_delays 7 2026-02-17T04:53:13.856 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.856 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.856 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: test '1 2 4 ' = '1 2 4 ' 2026-02-17T04:53:13.888 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: get_timeout_delays 8 2026-02-17T04:53:13.888 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.888 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.889 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.914 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: test '1 2 4 1 ' = '1 2 4 1 ' 2026-02-17T04:53:13.914 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: get_timeout_delays 1 .1 2026-02-17T04:53:13.915 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.915 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.915 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.915 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.915 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: test '0.1 0.2 0.4 0.3 ' = '0.1 0.2 0.4 0.3 ' 2026-02-17T04:53:13.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: get_timeout_delays 1.5 .1 2026-02-17T04:53:13.949 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.949 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:13.989 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 ' = '0.1 0.2 0.4 0.8 ' 2026-02-17T04:53:13.990 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: get_timeout_delays 5 .1 2026-02-17T04:53:13.991 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:13.991 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:13.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:13.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:13.991 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:14.040 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 1.9 ' = '0.1 0.2 0.4 0.8 1.6 1.9 ' 2026-02-17T04:53:14.041 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: get_timeout_delays 6 .1 2026-02-17T04:53:14.041 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:14.041 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:14.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:14.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:14.042 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:14.091 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 2.9 ' = '0.1 0.2 0.4 0.8 1.6 2.9 ' 2026-02-17T04:53:14.092 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: get_timeout_delays 6.3 .1 2026-02-17T04:53:14.092 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:14.092 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:14.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:14.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:14.093 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:14.143 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 ' = '0.1 0.2 0.4 0.8 1.6 3.2 ' 2026-02-17T04:53:14.143 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: get_timeout_delays 20 .1 2026-02-17T04:53:14.144 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:14.144 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:14.144 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:14.144 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:14.145 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:14.211 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' 2026-02-17T04:53:14.212 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: get_timeout_delays 300 .1 0 2026-02-17T04:53:14.212 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:14.212 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:14.212 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:14.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:14.213 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:14.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' 2026-02-17T04:53:14.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: get_timeout_delays 300 .1 10 2026-02-17T04:53:14.316 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:14.316 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:14.317 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:14.317 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:14.317 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_is_clean td/ceph-helpers 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1552: test_is_clean: local dir=td/ceph-helpers 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1554: test_is_clean: setup td/ceph-helpers 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:14.629 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:14.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:14.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:14.631 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:14.631 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:14.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:14.632 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:14.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:14.634 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:14.634 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:14.635 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:14.635 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:14.635 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:14.636 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:14.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:14.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:14.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:14.639 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:14.639 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.639 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.639 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:14.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:14.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:14.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:53:14.642 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:53:14.642 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.642 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.642 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:53:14.643 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1555: test_is_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:53:14.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:53:14.645 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:53:14.645 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.667 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:14.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:14.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:53:14.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:53:14.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:14.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:14.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:53:14.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.700 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:14.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:14.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:53:14.754 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:14.755 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:14.756 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:14.756 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.756 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.756 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:14.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:14.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:53:14.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1556: test_is_clean: run_mgr td/ceph-helpers x 2026-02-17T04:53:14.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:53:14.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:53:14.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:53:14.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:53:14.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:53:14.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.923 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:14.924 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:14.925 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1557: test_is_clean: run_osd td/ceph-helpers 0 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:53:14.946 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:14.947 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:53:14.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:53:14.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:53:14.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:53:14.949 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:53:14.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5b37812d-4f57-41dc-9bb8-39d3c6dcbaa2 2026-02-17T04:53:14.950 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5b37812d-4f57-41dc-9bb8-39d3c6dcbaa2' 2026-02-17T04:53:14.950 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 5b37812d-4f57-41dc-9bb8-39d3c6dcbaa2 2026-02-17T04:53:14.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:53:14.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA69JNprJYPORAA6EV0kZ6oM5WwjbLrgd7EOQ== 2026-02-17T04:53:14.959 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA69JNprJYPORAA6EV0kZ6oM5WwjbLrgd7EOQ=="}' 2026-02-17T04:53:14.960 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5b37812d-4f57-41dc-9bb8-39d3c6dcbaa2 -i td/ceph-helpers/0/new.json 2026-02-17T04:53:15.074 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:15.082 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:53:15.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA69JNprJYPORAA6EV0kZ6oM5WwjbLrgd7EOQ== --osd-uuid 5b37812d-4f57-41dc-9bb8-39d3c6dcbaa2 2026-02-17T04:53:15.100 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.094+0000 7f6b9a65da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:15.101 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.098+0000 7f6b9a65da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:15.102 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.098+0000 7f6b9a65da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:15.103 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.098+0000 7f6b9a65da40 -1 bdev(0x56335254a400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:53:15.103 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.098+0000 7f6b9a65da40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:53:15.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:53:15.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:53:15.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:53:15.772 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:53:15.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:53:15.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:53:15.887 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:53:15.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:53:15.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:53:15.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:53:15.887 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:53:15.903 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.898+0000 7f18c8ee9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:15.904 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.898+0000 7f18c8ee9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:15.905 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:15.902+0000 7f18c8ee9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:15.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:53:15.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:53:15.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:53:16.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:53:16.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:53:16.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:16.000 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:53:16.000 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:16.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:16.001 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:16.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:16.227 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:16.222+0000 7f18c8ee9a40 -1 Falling back to public interface 2026-02-17T04:53:17.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:17.110 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:17.110 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:53:17.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:53:17.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:17.111 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:17.218 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:17.311 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:17.306+0000 7f18c8ee9a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:53:18.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:18.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:18.220 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:53:18.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:53:18.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:18.220 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:18.441 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3985652495,v1:127.0.0.1:6801/3985652495] [v2:127.0.0.1:6802/3985652495,v1:127.0.0.1:6803/3985652495] exists,up 5b37812d-4f57-41dc-9bb8-39d3c6dcbaa2 2026-02-17T04:53:18.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:18.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:18.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:18.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1558: test_is_clean: create_rbd_pool 2026-02-17T04:53:18.442 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:53:18.648 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:53:18.658 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:53:18.658 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:53:18.886 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:53:18.924 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:53:19.897 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:53:20.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1559: test_is_clean: wait_for_clean 2026-02-17T04:53:20.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:53:20.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:53:20.178 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:53:20.178 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:53:20.179 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:20.179 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:20.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:20.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:20.179 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:20.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:53:20.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:53:20.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:53:20.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:53:20.281 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:53:20.281 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:53:20.514 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:53:20.514 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:53:20.514 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:53:20.514 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:53:20.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:53:20.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:53:20.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:53:20.592 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:53:20.592 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:20.592 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:53:20.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:53:20.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:20.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:53:20.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:53:20.596 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:53:20.596 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:53:20.596 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:20.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:53:20.819 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:53:21.821 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:53:21.821 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:22.032 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:53:22.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:53:22.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:22.033 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:22.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:53:22.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:53:22.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:53:22.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:53:22.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:53:22.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:53:22.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:53:22.314 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:53:22.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:53:22.527 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:53:22.527 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:22.527 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:22.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:53:22.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:53:22.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:53:22.809 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1560: test_is_clean: is_clean 2026-02-17T04:53:22.810 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: get_num_pgs 2026-02-17T04:53:22.810 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:22.810 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:23.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: num_pgs=4 2026-02-17T04:53:23.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1547: is_clean: test 4 '!=' 0 2026-02-17T04:53:23.099 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: get_num_active_clean 2026-02-17T04:53:23.099 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:53:23.099 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:53:23.099 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:53:23.100 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:53:23.100 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:53:23.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: test 4 = 4 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1561: test_is_clean: teardown td/ceph-helpers 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:23.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:23.424 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:23.425 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:23.425 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:23.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:23.427 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:23.427 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:23.427 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:23.428 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:23.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:23.429 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:23.429 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:23.430 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:23.431 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:23.431 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:23.435 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:23.435 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.435 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.435 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemon td/ceph-helpers 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: test_kill_daemon: local dir=td/ceph-helpers 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:267: test_kill_daemon: setup td/ceph-helpers 2026-02-17T04:53:23.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:23.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:23.440 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:23.440 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:23.441 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:23.441 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:23.443 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:23.443 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:23.443 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:23.444 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:23.444 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:23.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:23.445 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:23.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:23.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:23.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:23.448 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:23.448 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.448 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.448 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:23.449 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:23.450 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:23.450 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:53:23.451 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:53:23.451 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.451 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.451 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:53:23.452 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:53:23.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:53:23.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:53:23.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:53:23.454 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: test_kill_daemon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:23.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:53:23.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:53:23.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:53:23.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:53:23.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:53:23.455 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:23.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:53:23.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:23.475 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:23.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:23.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.475 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.476 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:23.476 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:23.506 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:53:23.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:53:23.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:23.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:23.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:53:23.507 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:23.509 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:53:23.564 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:53:23.565 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:23.565 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:23.565 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:53:23.565 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:23.566 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:53:23.623 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: test_kill_daemon: run_mgr td/ceph-helpers x 2026-02-17T04:53:23.623 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:53:23.623 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:53:23.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:53:23.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:53:23.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:53:23.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:53:23.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:53:23.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:23.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:23.737 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:23.738 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.738 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:23.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:23.739 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:23.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:270: test_kill_daemon: run_osd td/ceph-helpers 0 2026-02-17T04:53:23.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:53:23.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:53:23.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:53:23.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:53:23.759 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:23.760 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:53:23.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:53:23.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:53:23.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:53:23.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:53:23.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:53:23.762 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:53:23.762 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3f3b096e-d7af-47b5-98ce-33b48f83d38a 2026-02-17T04:53:23.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3f3b096e-d7af-47b5-98ce-33b48f83d38a' 2026-02-17T04:53:23.763 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 3f3b096e-d7af-47b5-98ce-33b48f83d38a 2026-02-17T04:53:23.763 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:53:23.773 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBD9JNpQRfyLRAATUizZEI8SFjJK9cLb00VSg== 2026-02-17T04:53:23.773 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBD9JNpQRfyLRAATUizZEI8SFjJK9cLb00VSg=="}' 2026-02-17T04:53:23.773 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3f3b096e-d7af-47b5-98ce-33b48f83d38a -i td/ceph-helpers/0/new.json 2026-02-17T04:53:23.884 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:23.892 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:53:23.893 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBD9JNpQRfyLRAATUizZEI8SFjJK9cLb00VSg== --osd-uuid 3f3b096e-d7af-47b5-98ce-33b48f83d38a 2026-02-17T04:53:23.908 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:23.902+0000 7f63beb3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:23.910 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:23.906+0000 7f63beb3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:23.911 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:23.906+0000 7f63beb3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:23.911 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:23.906+0000 7f63beb3fa40 -1 bdev(0x5595814dc400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:53:23.911 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:23.906+0000 7f63beb3fa40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:53:24.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:53:24.624 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:53:24.625 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:53:24.625 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:53:24.626 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:53:24.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:53:24.752 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:53:24.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:53:24.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:53:24.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:53:24.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:53:24.775 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:24.770+0000 7f36ccbc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:24.777 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:24.770+0000 7f36ccbc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:24.779 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:24.774+0000 7f36ccbc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:24.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:53:24.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:53:24.864 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:53:24.865 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:53:24.865 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:53:24.865 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:24.865 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:53:24.865 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:24.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:24.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:24.978 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:25.171 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:25.166+0000 7f36ccbc1a40 -1 Falling back to public interface 2026-02-17T04:53:25.868 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:25.862+0000 7f36ccbc1a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:53:25.979 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:25.979 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:25.979 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:53:25.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:53:25.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:25.980 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:26.090 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:27.091 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:27.091 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:27.091 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:53:27.091 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:53:27.092 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:27.092 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:27.315 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3921845221,v1:127.0.0.1:6801/3921845221] [v2:127.0.0.1:6802/3921845221,v1:127.0.0.1:6803/3921845221] exists,up 3f3b096e-d7af-47b5-98ce-33b48f83d38a 2026-02-17T04:53:27.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:27.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:27.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:27.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:272: test_kill_daemon: name_prefix=osd 2026-02-17T04:53:27.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: find td/ceph-helpers 2026-02-17T04:53:27.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: grep 'osd[^/]*\.pid' 2026-02-17T04:53:27.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-17T04:53:27.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:280: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid 0 1 2026-02-17T04:53:27.318 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=97247 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=0 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local delays=1 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 97247 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:27.319 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:27.320 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 1 2026-02-17T04:53:28.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 97247 2026-02-17T04:53:28.321 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:28.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:28.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0 2026-02-17T04:53:28.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 1 2026-02-17T04:53:28.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:284: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid TERM 2026-02-17T04:53:28.323 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-17T04:53:28.324 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=97247 2026-02-17T04:53:28.324 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 97247 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:28.325 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-17T04:53:28.425 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 97247 2026-02-17T04:53:28.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:28.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:28.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.2 2026-02-17T04:53:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 97247 2026-02-17T04:53:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-17T04:53:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-17T04:53:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-17T04:53:28.627 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:287: test_kill_daemon: name_prefix=mgr 2026-02-17T04:53:28.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: find td/ceph-helpers 2026-02-17T04:53:28.628 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: grep 'mgr[^/]*\.pid' 2026-02-17T04:53:28.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-17T04:53:28.630 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:292: test_kill_daemon: kill_daemon td/ceph-helpers/mgr.x.pid TERM 2026-02-17T04:53:28.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mgr.x.pid 2026-02-17T04:53:28.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=96391 2026-02-17T04:53:28.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-17T04:53:28.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-17T04:53:28.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T04:53:28.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 96391 2026-02-17T04:53:28.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:28.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:28.633 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-17T04:53:28.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 96391 2026-02-17T04:53:28.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-17T04:53:28.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-17T04:53:28.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-17T04:53:28.735 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:295: test_kill_daemon: name_prefix=mon 2026-02-17T04:53:28.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: find td/ceph-helpers 2026-02-17T04:53:28.735 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: grep 'mon[^/]*\.pid' 2026-02-17T04:53:28.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-17T04:53:28.736 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:300: test_kill_daemon: kill_daemon td/ceph-helpers/mon.a.pid TERM 2026-02-17T04:53:28.736 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mon.a.pid 2026-02-17T04:53:28.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=96306 2026-02-17T04:53:28.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-17T04:53:28.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-17T04:53:28.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T04:53:28.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.738 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 96306 2026-02-17T04:53:28.739 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:28.739 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:28.739 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-17T04:53:28.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:28.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 96306 2026-02-17T04:53:28.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T04:53:28.839 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T04:53:28.840 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.2 2026-02-17T04:53:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T04:53:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 96306 2026-02-17T04:53:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-17T04:53:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-17T04:53:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-17T04:53:29.041 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:301: test_kill_daemon: timeout 5 ceph status 2026-02-17T04:53:34.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:304: test_kill_daemon: teardown td/ceph-helpers 2026-02-17T04:53:34.045 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:34.046 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:34.046 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:34.046 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:34.046 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:34.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:34.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:34.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:34.052 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:34.052 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:34.053 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:34.053 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:34.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:34.054 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:34.055 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:34.056 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:34.056 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:34.056 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:34.057 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:34.058 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:34.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:34.059 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:34.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:34.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.063 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.064 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemons td/ceph-helpers 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:357: test_kill_daemons: local dir=td/ceph-helpers 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:358: test_kill_daemons: setup td/ceph-helpers 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:34.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:34.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:34.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:34.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:34.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:34.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:34.068 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:34.068 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:34.069 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:34.070 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:34.071 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:34.071 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:34.071 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:34.072 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:34.072 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:34.073 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:34.073 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:34.074 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:34.075 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:34.075 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:34.076 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:34.076 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.076 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:34.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:34.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:34.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:53:34.080 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:53:34.080 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.080 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.080 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:53:34.081 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:53:34.082 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:359: test_kill_daemons: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:53:34.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:34.106 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:53:34.106 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:34.106 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:34.106 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:34.106 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.106 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.107 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:34.107 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:34.137 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:53:34.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:53:34.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:34.138 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:34.139 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:53:34.139 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:53:34.139 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:34.139 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:34.139 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:34.140 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:34.140 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.140 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.140 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:34.140 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:34.140 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:53:34.196 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:53:34.196 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.197 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.198 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:34.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:34.198 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:53:34.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:360: test_kill_daemons: run_mgr td/ceph-helpers x 2026-02-17T04:53:34.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:53:34.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:53:34.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:53:34.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:53:34.253 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:53:34.254 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:53:34.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:53:34.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:34.367 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:34.367 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:34.367 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.368 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.368 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:34.368 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:34.369 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:34.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:361: test_kill_daemons: run_osd td/ceph-helpers 0 2026-02-17T04:53:34.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:53:34.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:53:34.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:53:34.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:53:34.390 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:53:34.391 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:53:34.392 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:53:34.393 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:53:34.393 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:53:34.393 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:53:34.393 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:53:34.393 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:53:34.393 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:53:34.394 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3f77fcc1-17bc-4560-9a1a-2036e8cd5b6e 2026-02-17T04:53:34.394 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3f77fcc1-17bc-4560-9a1a-2036e8cd5b6e' 2026-02-17T04:53:34.394 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 3f77fcc1-17bc-4560-9a1a-2036e8cd5b6e 2026-02-17T04:53:34.394 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:53:34.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBO9JNpjBjaFxAAy5To0eri8fKPcVvMO9zipQ== 2026-02-17T04:53:34.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBO9JNpjBjaFxAAy5To0eri8fKPcVvMO9zipQ=="}' 2026-02-17T04:53:34.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3f77fcc1-17bc-4560-9a1a-2036e8cd5b6e -i td/ceph-helpers/0/new.json 2026-02-17T04:53:34.512 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:34.520 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:53:34.521 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBO9JNpjBjaFxAAy5To0eri8fKPcVvMO9zipQ== --osd-uuid 3f77fcc1-17bc-4560-9a1a-2036e8cd5b6e 2026-02-17T04:53:34.537 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:34.530+0000 7fe83a943a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:34.538 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:34.534+0000 7fe83a943a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:34.539 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:34.534+0000 7fe83a943a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:34.539 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:34.534+0000 7fe83a943a40 -1 bdev(0x56328daac400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:53:34.540 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:34.534+0000 7fe83a943a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:53:35.314 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:53:35.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:53:35.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:53:35.316 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:53:35.316 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:53:35.440 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:53:35.441 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:53:35.441 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:53:35.441 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:53:35.441 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:53:35.441 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:53:35.466 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:35.462+0000 7f0a8a9a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:35.468 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:35.462+0000 7f0a8a9a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:35.469 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:35.462+0000 7f0a8a9a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:35.549 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:53:35.550 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:53:35.550 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:53:35.550 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:53:35.550 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:53:35.550 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:35.550 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:35.551 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:53:35.551 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:35.551 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:35.664 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:35.999 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:35.994+0000 7f0a8a9a5a40 -1 Falling back to public interface 2026-02-17T04:53:36.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:36.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:36.666 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:53:36.666 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:53:36.666 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:36.666 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:36.723 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:36.718+0000 7f0a8a9a5a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:53:36.776 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:37.778 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:37.778 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:37.779 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:53:37.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:53:37.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:37.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:37.998 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3590299052,v1:127.0.0.1:6801/3590299052] [v2:127.0.0.1:6802/3590299052,v1:127.0.0.1:6803/3590299052] exists,up 3f77fcc1-17bc-4560-9a1a-2036e8cd5b6e 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:368: test_kill_daemons: kill_daemons td/ceph-helpers 0 osd 1 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:37.999 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:39.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 1 2026-02-17T04:53:39.007 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:372: test_kill_daemons: kill_daemons td/ceph-helpers TERM osd 2026-02-17T04:53:39.008 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:39.008 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:39.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:39.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:39.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:39.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:39.317 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:376: test_kill_daemons: kill_daemons td/ceph-helpers TERM mgr 2026-02-17T04:53:39.318 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:39.318 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:39.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:39.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:39.318 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:380: test_kill_daemons: kill_daemons td/ceph-helpers TERM 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:39.426 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:39.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:39.734 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:381: test_kill_daemons: timeout 5 ceph status 2026-02-17T04:53:44.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:382: test_kill_daemons: teardown td/ceph-helpers 2026-02-17T04:53:44.742 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:44.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:44.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:44.743 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:44.743 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:44.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:44.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:44.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:44.748 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:44.748 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:44.749 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:44.750 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:44.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:44.751 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:44.751 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:44.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:44.752 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:44.753 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:44.753 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:44.754 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:44.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:44.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:44.760 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:44.760 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:44.761 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:44.761 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:44.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:44.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:44.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:44.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_objectstore_tool td/ceph-helpers 2026-02-17T04:53:44.763 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: test_objectstore_tool: local dir=td/ceph-helpers 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: test_objectstore_tool: setup td/ceph-helpers 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:44.764 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:44.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:44.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:44.765 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:44.766 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:44.766 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:44.767 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:44.767 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:44.768 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:44.768 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:44.768 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:44.769 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:44.769 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:44.770 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:44.770 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:44.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:44.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:44.772 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:44.773 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:44.773 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:44.773 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:44.774 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:44.775 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:44.775 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:44.775 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:53:44.776 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:53:44.776 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:44.776 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:44.776 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:53:44.778 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1301: test_objectstore_tool: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:53:44.779 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:53:44.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:53:44.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:53:44.780 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:44.802 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:44.803 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:44.830 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:53:44.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:53:44.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:44.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:44.831 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:53:44.833 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:53:44.833 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:44.833 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:44.833 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:44.833 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:44.833 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:44.834 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:44.834 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:44.834 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:44.834 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:53:44.891 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:44.892 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:44.893 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:44.893 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:44.893 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:44.894 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:44.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:44.894 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:53:44.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: test_objectstore_tool: local osd=0 2026-02-17T04:53:44.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: test_objectstore_tool: run_mgr td/ceph-helpers x 2026-02-17T04:53:44.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:53:44.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:53:44.948 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:53:44.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:53:44.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:53:44.949 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:53:45.061 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:45.062 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:45.063 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:53:45.085 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: test_objectstore_tool: run_osd td/ceph-helpers 0 2026-02-17T04:53:45.085 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:53:45.085 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:53:45.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:53:45.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:53:45.088 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:53:45.089 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:53:45.089 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2625c544-8660-4b3e-bb93-0de87ce03a4e 2026-02-17T04:53:45.089 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2625c544-8660-4b3e-bb93-0de87ce03a4e' 2026-02-17T04:53:45.089 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 2625c544-8660-4b3e-bb93-0de87ce03a4e 2026-02-17T04:53:45.090 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:53:45.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZ9JNpybjEBRAAahc/SK6HMKLVEko2hgugJQ== 2026-02-17T04:53:45.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZ9JNpybjEBRAAahc/SK6HMKLVEko2hgugJQ=="}' 2026-02-17T04:53:45.099 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2625c544-8660-4b3e-bb93-0de87ce03a4e -i td/ceph-helpers/0/new.json 2026-02-17T04:53:45.210 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:45.217 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:53:45.219 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZ9JNpybjEBRAAahc/SK6HMKLVEko2hgugJQ== --osd-uuid 2625c544-8660-4b3e-bb93-0de87ce03a4e 2026-02-17T04:53:45.235 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:45.229+0000 7fa5117c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:45.238 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:45.233+0000 7fa5117c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:45.239 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:45.233+0000 7fa5117c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:45.239 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:45.233+0000 7fa5117c6a40 -1 bdev(0x558c1cd44400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:53:45.240 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:45.233+0000 7fa5117c6a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:53:45.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:53:45.974 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:53:45.975 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:53:45.975 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:53:45.976 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:53:46.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:53:46.100 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:53:46.101 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:53:46.101 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:53:46.101 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:53:46.101 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:53:46.126 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:46.121+0000 7feacc366a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:46.128 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:46.121+0000 7feacc366a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:46.129 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:46.121+0000 7feacc366a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:46.212 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:53:46.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:53:46.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:53:46.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:53:46.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:53:46.213 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:46.213 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:53:46.214 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:53:46.214 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:46.214 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:46.322 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:46.588 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:46.581+0000 7feacc366a40 -1 Falling back to public interface 2026-02-17T04:53:47.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:47.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:47.323 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:53:47.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:53:47.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:47.323 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:47.436 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:47.550 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:47.545+0000 7feacc366a40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:53:48.437 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:48.438 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:48.438 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:53:48.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:53:48.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:48.439 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:48.648 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1105300927,v1:127.0.0.1:6801/1105300927] [v2:127.0.0.1:6802/1105300927,v1:127.0.0.1:6803/1105300927] exists,up 2625c544-8660-4b3e-bb93-0de87ce03a4e 2026-02-17T04:53:48.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:48.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:48.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:48.649 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: test_objectstore_tool: create_rbd_pool 2026-02-17T04:53:48.650 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:53:48.852 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:53:48.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:53:48.861 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:53:49.135 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:53:49.144 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:53:50.146 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: test_objectstore_tool: wait_for_clean 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:50.445 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:50.446 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:50.446 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:50.446 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:50.535 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:53:50.536 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:53:50.536 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:53:50.536 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:53:50.536 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:53:50.536 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:53:50.745 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:53:50.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:53:50.746 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:53:50.746 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:53:50.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:53:50.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:53:50.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:53:50.828 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:53:50.829 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:50.829 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:53:50.829 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:53:50.830 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:53:50.830 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:53:50.831 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:53:50.832 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:53:50.832 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:53:50.832 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:51.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:53:51.047 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:53:52.048 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:53:52.049 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:52.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:53:52.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:53:52.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:52.269 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:53:52.563 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:53:52.771 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:53:52.772 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:53:52.772 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:52.772 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:53.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:53:53.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:53:53.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:53:53.057 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1307: test_objectstore_tool: rados --pool rbd put GROUP /etc/group 2026-02-17T04:53:53.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1308: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 GROUP get-bytes 2026-02-17T04:53:53.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-17T04:53:53.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-17T04:53:53.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-17T04:53:53.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-17T04:53:53.077 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 GROUP get-bytes 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1309: test_objectstore_tool: diff - /etc/group 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:53.078 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:53.079 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:53.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:53.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 GROUP get-bytes 2026-02-17T04:53:53.385 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-17T04:53:53.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-17T04:53:53.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-17T04:53:53.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-17T04:53:53.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:53.386 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 GROUP get-bytes 2026-02-17T04:53:53.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/ceph-helpers 0 2026-02-17T04:53:53.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T04:53:53.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T04:53:53.524 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:53:53.525 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:53:53.526 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T04:53:53.527 INFO:tasks.workunit.client.0.trial105.stderr:start osd.0 2026-02-17T04:53:53.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:53:53.528 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T04:53:53.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T04:53:53.528 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T04:53:53.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T04:53:53.529 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T04:53:53.542 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:53.537+0000 7f2c72847a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:53.544 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:53.537+0000 7f2c72847a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:53.545 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:53.537+0000 7f2c72847a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:53:53.743 INFO:tasks.workunit.client.0.trial105.stderr:0 2026-02-17T04:53:53.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:53.744 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:53.957 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:53:53.983 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:53.977+0000 7f2c72847a40 -1 Falling back to public interface 2026-02-17T04:53:54.621 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:53:54.613+0000 7f2c72847a40 -1 osd.0 13 log_to_monitors true 2026-02-17T04:53:54.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:53:54.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:53:54.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:53:54.958 INFO:tasks.workunit.client.0.trial105.stderr:1 2026-02-17T04:53:54.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:53:54.958 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:osd.0 up in weight 1 up_from 16 up_thru 16 down_at 13 last_clean_interval [5,13) [v2:127.0.0.1:6800/350379494,v1:127.0.0.1:6801/350379494] [v2:127.0.0.1:6802/350379494,v1:127.0.0.1:6803/350379494] exists,up 2625c544-8660-4b3e-bb93-0de87ce03a4e 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:53:55.171 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:53:55.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:53:55.172 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:53:55.172 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:53:55.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:53:55.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:53:55.172 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:53:55.273 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:53:55.274 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:53:55.274 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:53:55.274 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:53:55.274 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:53:55.274 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:53:55.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:53:55.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:53:55.490 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:53:55.490 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:53:55.571 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=68719476739 2026-02-17T04:53:55.572 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 68719476739 2026-02-17T04:53:55.572 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-68719476739' 2026-02-17T04:53:55.572 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:53:55.572 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-68719476739 2026-02-17T04:53:55.572 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:53:55.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:53:55.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-68719476739 2026-02-17T04:53:55.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:53:55.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=68719476739 2026-02-17T04:53:55.575 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 68719476739' 2026-02-17T04:53:55.575 INFO:tasks.workunit.client.0.trial105.stderr:waiting osd.0 seq 68719476739 2026-02-17T04:53:55.575 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:55.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 68719476739 2026-02-17T04:53:55.797 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:53:56.798 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:53:56.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:57.013 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 68719476739 2026-02-17T04:53:57.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:53:58.014 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T04:53:58.015 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:53:58.233 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 68719476739 -lt 68719476739 2026-02-17T04:53:58.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:53:58.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:58.234 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:58.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:53:58.516 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:53:58.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:53:58.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:53:58.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:53:58.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:53:58.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:53:58.517 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:53:58.725 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:53:58.725 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:53:58.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:53:58.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:53:59.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:53:59.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:53:59.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:53:59.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1310: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-17T04:53:59.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-17T04:53:59.004 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:59.005 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:59.006 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:59.312 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:59.312 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-17T04:53:59.312 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-17T04:53:59.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-17T04:53:59.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-17T04:53:59.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-17T04:53:59.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-17T04:53:59.313 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 NOTEXISTS get-bytes 2026-02-17T04:53:59.376 INFO:tasks.workunit.client.0.trial105.stderr:No object id 'NOTEXISTS' found or invalid JSON specified 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: return 1 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: return 1 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1311: test_objectstore_tool: teardown td/ceph-helpers 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:59.467 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:59.468 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:59.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:59.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:59.468 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:59.576 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:59.577 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:59.577 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:59.578 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:59.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:59.579 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:59.580 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:59.581 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:59.581 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:59.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:59.582 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:59.582 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:59.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:59.584 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:59.590 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:59.590 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:59.590 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:59.591 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_schedule_scrub td/ceph-helpers 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1987: test_pg_schedule_scrub: local dir=td/ceph-helpers 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1989: test_pg_schedule_scrub: setup td/ceph-helpers 2026-02-17T04:53:59.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:53:59.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:53:59.595 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:53:59.595 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:53:59.596 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:53:59.597 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:53:59.598 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:53:59.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:53:59.599 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:53:59.599 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:59.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:53:59.600 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:53:59.600 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:53:59.601 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:53:59.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:53:59.602 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:53:59.603 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:53:59.604 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:59.604 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:59.604 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:53:59.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:53:59.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:53:59.605 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:53:59.606 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:53:59.606 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:59.606 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:59.607 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:53:59.608 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:53:59.609 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:53:59.609 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:53:59.609 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:53:59.609 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1990: test_pg_schedule_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:59.609 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:53:59.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:53:59.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:53:59.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:53:59.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:53:59.610 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:59.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:53:59.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:53:59.630 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:53:59.631 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:53:59.631 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:59.631 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:59.631 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:53:59.632 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:53:59.665 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:53:59.666 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:53:59.666 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:59.666 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:59.666 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:53:59.667 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:53:59.667 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:59.668 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:53:59.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:53:59.737 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:53:59.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:53:59.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:53:59.738 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:53:59.739 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:53:59.740 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:53:59.740 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:53:59.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1991: test_pg_schedule_scrub: run_mgr td/ceph-helpers x --mgr_stats_period=1 2026-02-17T04:53:59.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:53:59.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:53:59.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:53:59.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:53:59.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:53:59.795 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1992: test_pg_schedule_scrub: run_osd td/ceph-helpers 0 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:54:00.065 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b02af90a-f7c0-40ff-914b-8c5679637182 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b02af90a-f7c0-40ff-914b-8c5679637182' 2026-02-17T04:54:00.066 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 b02af90a-f7c0-40ff-914b-8c5679637182 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBn9JNpDbsVOBAAIItYM2Mw7HlYBuFVMI+U5A== 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBn9JNpDbsVOBAAIItYM2Mw7HlYBuFVMI+U5A=="}' 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b02af90a-f7c0-40ff-914b-8c5679637182 -i td/ceph-helpers/0/new.json 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:54:00.067 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBn9JNpDbsVOBAAIItYM2Mw7HlYBuFVMI+U5A== --osd-uuid b02af90a-f7c0-40ff-914b-8c5679637182 2026-02-17T04:54:00.079 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.073+0000 7fdfb7d3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:00.080 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.073+0000 7fdfb7d3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:00.081 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.073+0000 7fdfb7d3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:00.082 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.077+0000 7fdfb7d3ea40 -1 bdev(0x5605e0b96400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:54:00.082 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.077+0000 7fdfb7d3ea40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:54:00.792 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:54:00.793 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:54:00.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:54:00.794 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:54:00.794 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:54:00.906 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:54:00.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:54:00.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:54:00.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:54:00.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:54:00.907 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:54:00.920 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.913+0000 7fa107f5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:00.922 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.917+0000 7fa107f5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:00.923 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:00.917+0000 7fa107f5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:54:01.020 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:54:01.021 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:54:01.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:54:01.021 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:54:01.127 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:54:01.351 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:01.345+0000 7fa107f5ca40 -1 Falling back to public interface 2026-02-17T04:54:01.972 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:01.965+0000 7fa107f5ca40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:54:02.128 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:54:02.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:54:02.129 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:54:02.129 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:54:02.130 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:54:02.133 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:54:02.266 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:54:03.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:54:03.268 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:54:03.268 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:54:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:54:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:54:03.269 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:54:03.487 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2057403677,v1:127.0.0.1:6801/2057403677] [v2:127.0.0.1:6802/2057403677,v1:127.0.0.1:6803/2057403677] exists,up b02af90a-f7c0-40ff-914b-8c5679637182 2026-02-17T04:54:03.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:54:03.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:54:03.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:54:03.488 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1993: test_pg_schedule_scrub: create_rbd_pool 2026-02-17T04:54:03.489 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:54:03.691 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:54:03.702 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:54:03.702 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:54:03.944 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:54:03.955 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:54:04.956 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:54:05.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1994: test_pg_schedule_scrub: wait_for_clean 2026-02-17T04:54:05.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:54:05.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:54:05.235 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:54:05.235 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:54:05.236 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:54:05.236 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:54:05.236 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:54:05.236 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:54:05.236 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:54:05.341 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:54:05.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:54:05.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:54:05.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:54:05.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:54:05.342 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:54:05.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:54:05.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:54:05.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:54:05.553 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:54:05.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-17T04:54:05.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-17T04:54:05.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T04:54:05.637 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:54:05.638 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-17T04:54:05.638 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:54:05.638 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:54:05.639 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-17T04:54:05.639 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:54:05.640 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-17T04:54:05.641 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836485 2026-02-17T04:54:05.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T04:54:05.641 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:05.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836485 2026-02-17T04:54:05.878 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:54:06.879 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:54:06.880 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:07.096 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T04:54:07.096 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:54:07.097 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:54:07.097 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:54:07.384 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:54:07.597 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:54:07.597 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:54:07.597 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:54:07.597 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1995: test_pg_schedule_scrub: pg_schedule_scrub 1.0 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1969: pg_schedule_scrub: local pgid=1.0 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: pg_schedule_scrub: wait_for_pg_clean 1.0 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-17T04:54:07.885 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T04:54:07.886 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:54:07.886 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:54:07.888 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:54:07.888 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:54:07.888 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:54:08.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T04:54:08.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-17T04:54:08.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-17T04:54:08.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-17T04:54:08.150 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:54:08.150 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:54:08.364 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:54:08.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:54:08.365 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:54:08.365 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:54:08.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-17T04:54:08.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-17T04:54:08.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T04:54:08.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:54:08.447 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-17T04:54:08.447 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:54:08.447 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:54:08.448 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-17T04:54:08.448 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:54:08.449 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-17T04:54:08.450 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836490 2026-02-17T04:54:08.450 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T04:54:08.450 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:08.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836490 2026-02-17T04:54:08.668 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:54:09.669 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:54:09.670 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:09.881 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836490 2026-02-17T04:54:09.882 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:54:10.883 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T04:54:10.883 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:11.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-17T04:54:11.108 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stdout:#---------- 1.0 loop 0 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T04:54:11.109 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T04:54:11.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: get_last_scrub_stamp 1.0 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:11.192 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:11.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: local last_scrub=2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:11.403 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: pg_schedule_scrub: ceph tell 1.0 schedule-scrub 2026-02-17T04:54:11.478 INFO:tasks.workunit.client.0.trial105.stdout:{ 2026-02-17T04:54:11.478 INFO:tasks.workunit.client.0.trial105.stdout: "deep": false, 2026-02-17T04:54:11.478 INFO:tasks.workunit.client.0.trial105.stdout: "must": false, 2026-02-17T04:54:11.478 INFO:tasks.workunit.client.0.trial105.stdout: "stamp": "2026-02-14T16:54:11.476925+0000" 2026-02-17T04:54:11.478 INFO:tasks.workunit.client.0.trial105.stdout:} 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: pg_schedule_scrub: wait_for_scrub 1.0 2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:11.487 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:11.488 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:11.488 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:11.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:03.889040+0000 '>' 2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:11.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T04:54:12.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T04:54:12.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:12.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:12.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:12.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:12.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:12.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:12.908 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:03.889040+0000 '>' 2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:12.908 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T04:54:13.909 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T04:54:13.909 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:13.910 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:13.910 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:13.910 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:13.910 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:13.910 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:14.123 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:03.889040+0000 '>' 2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:14.123 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T04:54:15.124 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T04:54:15.125 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:15.125 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:15.125 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:15.125 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:15.126 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:15.126 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:15.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-14T16:54:11.476925+0000 '>' 2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:15.338 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T04:54:16.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T04:54:16.339 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:16.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:16.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:16.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:16.340 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:16.341 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:16.565 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:15.034195+0000 '>' 2026-02-17T04:54:03.889040+0000 2026-02-17T04:54:16.565 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-17T04:54:16.565 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1996: test_pg_schedule_scrub: teardown td/ceph-helpers 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:54:16.566 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:54:16.673 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:54:16.673 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:54:16.674 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:54:16.674 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:54:16.676 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:54:16.676 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:54:16.676 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:54:16.677 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:54:16.678 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:54:16.678 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:54:16.678 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:54:16.679 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:54:16.680 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:54:16.680 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:54:16.686 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:54:16.686 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.686 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.686 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_scrub td/ceph-helpers 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: test_pg_scrub: local dir=td/ceph-helpers 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: test_pg_scrub: setup td/ceph-helpers 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T04:54:16.689 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:54:16.690 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:54:16.691 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T04:54:16.691 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T04:54:16.692 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T04:54:16.693 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T04:54:16.693 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T04:54:16.693 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T04:54:16.694 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:54:16.694 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T04:54:16.695 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T04:54:16.695 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T04:54:16.696 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T04:54:16.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T04:54:16.697 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T04:54:16.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T04:54:16.698 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.699 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.699 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T04:54:16.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T04:54:16.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T04:54:16.700 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T04:54:16.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T04:54:16.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.701 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.701 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56401 2026-02-17T04:54:16.702 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: test_pg_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T04:54:16.704 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T04:54:16.705 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T04:54:16.705 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:54:16.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T04:54:16.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:54:16.726 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:54:16.726 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:54:16.727 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.727 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.727 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:16.727 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T04:54:16.755 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T04:54:16.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T04:54:16.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:54:16.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:54:16.756 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T04:54:16.757 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:54:16.758 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get fsid 2026-02-17T04:54:16.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T04:54:16.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T04:54:16.813 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T04:54:16.814 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T04:54:16.814 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56401/ceph-mon.a.asok 2026-02-17T04:54:16.815 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T04:54:16.816 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56401/ceph-mon.a.asok config get mon_host 2026-02-17T04:54:16.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: test_pg_scrub: run_mgr td/ceph-helpers x 2026-02-17T04:54:16.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T04:54:16.870 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T04:54:16.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T04:54:16.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T04:54:16.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T04:54:16.871 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T04:54:16.984 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T04:54:16.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:54:16.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:54:16.985 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:54:16.985 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:16.985 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:16.985 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:16.986 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:54:16.986 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T04:54:17.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: test_pg_scrub: run_osd td/ceph-helpers 0 2026-02-17T04:54:17.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T04:54:17.008 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T04:54:17.009 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T04:54:17.010 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T04:54:17.011 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T04:54:17.012 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T04:54:17.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ae329823-bfe7-48a0-bc03-8462eecc6233 2026-02-17T04:54:17.012 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ae329823-bfe7-48a0-bc03-8462eecc6233' 2026-02-17T04:54:17.013 INFO:tasks.workunit.client.0.trial105.stdout:add osd0 ae329823-bfe7-48a0-bc03-8462eecc6233 2026-02-17T04:54:17.013 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T04:54:17.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB59JNpts4tARAAZrzv24EgjRod9BYI9Slj3A== 2026-02-17T04:54:17.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB59JNpts4tARAAZrzv24EgjRod9BYI9Slj3A=="}' 2026-02-17T04:54:17.022 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ae329823-bfe7-48a0-bc03-8462eecc6233 -i td/ceph-helpers/0/new.json 2026-02-17T04:54:17.132 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:54:17.140 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T04:54:17.141 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB59JNpts4tARAAZrzv24EgjRod9BYI9Slj3A== --osd-uuid ae329823-bfe7-48a0-bc03-8462eecc6233 2026-02-17T04:54:17.157 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:17.149+0000 7ff516db9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:17.160 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:17.153+0000 7ff516db9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:17.161 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:17.153+0000 7ff516db9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:17.162 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:17.153+0000 7ff516db9a40 -1 bdev(0x55bda125e400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T04:54:17.162 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:17.153+0000 7ff516db9a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T04:54:17.933 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T04:54:17.934 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T04:54:17.934 INFO:tasks.workunit.client.0.trial105.stdout:adding osd0 key to auth repository 2026-02-17T04:54:17.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T04:54:17.935 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T04:54:18.061 INFO:tasks.workunit.client.0.trial105.stdout:start osd.0 2026-02-17T04:54:18.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T04:54:18.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=1fac9e9f-bf22-4024-9eba-bad1d56b3801 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.56401/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T04:54:18.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T04:54:18.061 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T04:54:18.062 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T04:54:18.087 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:18.081+0000 7f868549ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:18.089 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:18.081+0000 7f868549ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:18.091 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:18.085+0000 7f868549ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T04:54:18.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T04:54:18.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T04:54:18.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T04:54:18.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T04:54:18.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T04:54:18.174 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:54:18.175 INFO:tasks.workunit.client.0.trial105.stdout:0 2026-02-17T04:54:18.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T04:54:18.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:54:18.175 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:54:18.284 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:54:18.496 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:18.489+0000 7f868549ba40 -1 Falling back to public interface 2026-02-17T04:54:19.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:54:19.285 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:54:19.286 INFO:tasks.workunit.client.0.trial105.stdout:1 2026-02-17T04:54:19.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T04:54:19.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:54:19.286 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:54:19.432 INFO:tasks.workunit.client.0.trial105.stderr:2026-02-17T04:54:19.424+0000 7f868549ba40 -1 osd.0 0 log_to_monitors true 2026-02-17T04:54:19.502 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T04:54:20.503 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T04:54:20.504 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T04:54:20.504 INFO:tasks.workunit.client.0.trial105.stdout:2 2026-02-17T04:54:20.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-17T04:54:20.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T04:54:20.505 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T04:54:20.716 INFO:tasks.workunit.client.0.trial105.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2860439955,v1:127.0.0.1:6801/2860439955] [v2:127.0.0.1:6802/2860439955,v1:127.0.0.1:6803/2860439955] exists,up ae329823-bfe7-48a0-bc03-8462eecc6233 2026-02-17T04:54:20.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T04:54:20.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T04:54:20.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T04:54:20.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1947: test_pg_scrub: create_rbd_pool 2026-02-17T04:54:20.717 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T04:54:20.927 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' does not exist 2026-02-17T04:54:20.936 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T04:54:20.937 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T04:54:21.180 INFO:tasks.workunit.client.0.trial105.stderr:pool 'rbd' already exists 2026-02-17T04:54:21.191 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T04:54:22.193 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1948: test_pg_scrub: wait_for_clean 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:54:22.472 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:54:22.473 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:54:22.473 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:54:22.473 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:54:22.573 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T04:54:22.574 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T04:54:22.574 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T04:54:22.574 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T04:54:22.574 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:54:22.574 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:54:22.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:54:22.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:54:22.790 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:54:22.790 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:54:22.872 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T04:54:22.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T04:54:22.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T04:54:22.873 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:54:22.873 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T04:54:22.873 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:54:22.874 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:54:22.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T04:54:22.874 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:54:22.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T04:54:22.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T04:54:22.876 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836483 2026-02-17T04:54:22.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:23.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T04:54:23.100 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:54:24.101 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:54:24.102 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:24.315 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T04:54:24.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T04:54:24.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:54:24.316 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:54:24.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T04:54:24.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T04:54:24.593 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T04:54:24.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T04:54:24.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T04:54:24.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T04:54:24.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T04:54:24.594 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T04:54:24.798 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T04:54:24.798 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T04:54:24.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T04:54:24.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1949: test_pg_scrub: pg_scrub 1.0 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T04:54:25.083 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:54:25.084 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:54:25.084 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:54:25.084 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:54:25.084 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:54:25.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T04:54:25.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-17T04:54:25.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-17T04:54:25.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-17T04:54:25.342 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T04:54:25.342 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:54:25.561 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:54:25.562 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:54:25.562 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:54:25.562 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T04:54:25.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-17T04:54:25.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-17T04:54:25.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T04:54:25.641 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T04:54:25.641 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836486 2026-02-17T04:54:25.641 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T04:54:25.642 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T04:54:25.643 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-17T04:54:25.643 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T04:54:25.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836486 2026-02-17T04:54:25.644 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T04:54:25.644 INFO:tasks.workunit.client.0.trial105.stdout:waiting osd.0 seq 21474836486 2026-02-17T04:54:25.645 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:25.865 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836486 2026-02-17T04:54:25.865 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:54:26.866 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T04:54:26.867 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:27.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836486 2026-02-17T04:54:27.086 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T04:54:28.087 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T04:54:28.087 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T04:54:28.301 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T04:54:28.302 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T04:54:28.302 INFO:tasks.workunit.client.0.trial105.stdout:#---------- 1.0 loop 0 2026-02-17T04:54:28.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T04:54:28.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T04:54:28.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T04:54:28.303 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T04:54:28.304 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T04:54:28.304 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T04:54:28.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-17T04:54:28.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T04:54:28.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-17T04:54:28.384 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-17T04:54:28.385 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T04:54:28.385 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:28.385 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:28.385 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:28.385 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:28.593 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-17T04:54:21.124373+0000 2026-02-17T04:54:28.594 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-17T04:54:28.798 INFO:tasks.workunit.client.0.trial105.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-17T04:54:28.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-17T04:54:21.124373+0000 2026-02-17T04:54:28.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-17T04:54:28.810 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-17T04:54:21.124373+0000 2026-02-17T04:54:28.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T04:54:28.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-17T04:54:28.811 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:28.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:28.811 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:28.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:28.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:28.812 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:29.017 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:21.124373+0000 '>' 2026-02-17T04:54:21.124373+0000 2026-02-17T04:54:29.017 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T04:54:30.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T04:54:30.019 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:30.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:30.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:30.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:30.019 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:30.020 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:30.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:21.124373+0000 '>' 2026-02-17T04:54:21.124373+0000 2026-02-17T04:54:30.229 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T04:54:31.230 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T04:54:31.231 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T04:54:31.231 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T04:54:31.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T04:54:31.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T04:54:31.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T04:54:31.232 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T04:54:31.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T04:54:29.233952+0000 '>' 2026-02-17T04:54:21.124373+0000 2026-02-17T04:54:31.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-17T04:54:31.445 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1950: test_pg_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-17T04:54:31.445 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T04:54:31.446 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T04:54:31.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T04:54:31.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T04:54:31.446 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T04:54:31.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T04:54:31.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: WAIT_FOR_CLEAN_TIMEOUT=10 2026-02-17T04:54:31.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: TIMEOUT=2 2026-02-17T04:54:31.553 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: pg_scrub 1.0 2026-02-17T04:54:31.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-17T04:54:31.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T04:54:31.554 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-17T04:54:31.554 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 10 1 3 2026-02-17T04:54:31.555 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T04:54:31.555 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T04:54:31.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T04:54:31.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T04:54:31.555 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T04:54:31.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '1') 2026-02-17T04:54:31.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-17T04:54:31.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-17T04:54:31.587 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-17T04:54:31.588 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=2 2026-02-17T04:54:31.588 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T04:54:31.799 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T04:54:31.799 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T04:54:31.799 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T04:54:31.799 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 2 ceph tell osd.0 flush_pg_stats 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T04:54:33.807 INFO:tasks.workunit.client.0.trial105.stdout:#---------- 1.0 loop 0 2026-02-17T04:54:33.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T04:54:33.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T04:54:33.808 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T04:54:33.808 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T04:54:33.808 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T07:50:58.852 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1: is_pg_clean: teardown td/ceph-helpers 1 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T07:50:58.876 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T07:50:58.877 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T07:50:58.877 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T07:50:58.950 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T07:50:58.951 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T07:50:58.952 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T07:50:58.952 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T07:50:58.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T07:50:58.954 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T07:50:58.954 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T07:50:58.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T07:50:58.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T07:50:58.956 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T07:50:58.957 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T07:50:58.958 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T07:50:58.959 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-17T07:50:58.959 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-17T07:50:58.959 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-17T07:50:58.960 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/ceph-helpers/mgr.x.log td/ceph-helpers/mon.a.log td/ceph-helpers/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-17T07:50:58.962 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T07:50:58.966 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T07:50:58.967 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T07:50:58.967 INFO:tasks.workunit.client.0.trial105.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56401 2026-02-17T07:50:58.967 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56401 2026-02-17T07:50:58.968 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T07:50:58.968 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T07:50:58.968 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T07:50:58.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 0 >= 5 )) 2026-02-17T07:50:58.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 1 2026-02-17T07:50:59.969 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-17T07:50:59.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T07:50:59.970 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 1' 2026-02-17T07:50:59.970 INFO:tasks.workunit.client.0.trial105.stdout:#---------- 1.0 loop 1 2026-02-17T07:50:59.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T07:50:59.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T07:50:59.971 INFO:tasks.workunit.client.0.trial105.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T07:50:59.971 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T07:50:59.971 INFO:tasks.workunit.client.0.trial105.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T07:51:58.876 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-17T07:52:58.877 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-17T07:52:58.878 INFO:tasks.workunit:Stopping ['misc'] on client.0... 2026-02-17T07:52:58.878 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T07:52:59.208 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test misc/test-ceph-helpers.sh) on trial105 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh' 2026-02-17T07:52:59.212 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T07:52:59.229 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T07:52:59.229 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T07:52:59.344 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T07:52:59.356 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T07:52:59.356 DEBUG:teuthology.orchestra.run.trial105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T07:52:59.409 INFO:teuthology.orchestra.run.trial105.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T07:52:59.409 INFO:teuthology.orchestra.run.trial105.stdout:============================================================================== 2026-02-17T07:52:59.410 INFO:teuthology.orchestra.run.trial105.stdout:*lax1.us.ntp.li 131.215.220.143 2 u 200 256 377 63.136 +0.455 1.257 2026-02-17T07:52:59.410 INFO:teuthology.orchestra.run.trial105.stdout:+baron.cgee.net 216.239.35.4 2 u 147 256 377 43.218 +0.073 0.473 2026-02-17T07:52:59.410 INFO:teuthology.orchestra.run.trial105.stdout:-h96-60-247-29.m 69.89.207.199 2 u 238 256 377 28.092 +2.576 3.393 2026-02-17T07:52:59.410 INFO:teuthology.orchestra.run.trial105.stdout:+ntp.maxhost.io 132.163.97.4 2 u 181 256 347 69.695 +0.766 4.765 2026-02-17T07:52:59.410 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T07:52:59.422 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T07:52:59.422 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T07:52:59.436 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T07:52:59.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T07:52:59.466 INFO:teuthology.task.internal:Duration was 11507.224694 seconds 2026-02-17T07:52:59.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T07:52:59.479 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T07:52:59.480 DEBUG:teuthology.orchestra.run.trial105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T07:52:59.504 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T07:52:59.504 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial105.front.sepia.ceph.com 2026-02-17T07:52:59.505 DEBUG:teuthology.orchestra.run.trial105:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T07:52:59.551 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T07:52:59.551 DEBUG:teuthology.orchestra.run.trial105:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T07:52:59.749 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T07:52:59.749 DEBUG:teuthology.orchestra.run.trial105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T07:52:59.759 INFO:teuthology.orchestra.run.trial105.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T07:52:59.760 INFO:teuthology.orchestra.run.trial105.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T07:52:59.760 INFO:teuthology.orchestra.run.trial105.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T07:52:59.760 INFO:teuthology.orchestra.run.trial105.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T07:52:59.760 INFO:teuthology.orchestra.run.trial105.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T07:52:59.806 INFO:teuthology.orchestra.run.trial105.stderr: 81.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T07:52:59.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T07:52:59.820 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T07:52:59.821 DEBUG:teuthology.orchestra.run.trial105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T07:52:59.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T07:52:59.870 DEBUG:teuthology.orchestra.run.trial105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T07:52:59.906 INFO:teuthology.orchestra.run.trial105.stdout:kernel.core_pattern = core 2026-02-17T07:52:59.914 DEBUG:teuthology.orchestra.run.trial105:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T07:52:59.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T07:52:59.961 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T07:52:59.974 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T07:52:59.977 DEBUG:teuthology.misc:Transferring archived files from trial105:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_03:53:03-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial/50889/remote/trial105 2026-02-17T07:52:59.979 DEBUG:teuthology.orchestra.run.trial105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T07:53:13.682 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T07:53:13.682 DEBUG:teuthology.orchestra.run.trial105:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T07:53:13.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T07:53:13.787 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T07:53:13.787 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T07:53:13.801 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T07:53:13.801 DEBUG:teuthology.orchestra.run.trial105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T07:53:13.805 INFO:teuthology.orchestra.run.trial105.stdout: 1180220 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 07:53 /home/ubuntu/cephtest 2026-02-17T07:53:13.806 INFO:teuthology.orchestra.run.trial105.stdout: 1190961 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 17 04:44 /home/ubuntu/cephtest/mnt.0 2026-02-17T07:53:13.806 INFO:teuthology.orchestra.run.trial105.stdout: 1190962 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 17 04:50 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T07:53:13.806 INFO:teuthology.orchestra.run.trial105.stdout: 1206874 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 17 04:50 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T07:53:13.806 INFO:teuthology.orchestra.run.trial105.stdout: 1206875 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 17 07:50 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-17T07:53:13.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T07:53:13.806 INFO:teuthology.orchestra.run.trial105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-17T07:53:13.807 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial105 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T07:53:13.808 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T07:53:13.821 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T07:53:13.838 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial105 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T07:53:13.843 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} duration: 11507.224693775177 failure_reason: 'Command failed (workunit test misc/test-ceph-helpers.sh) on trial105 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh''' flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-17T07:53:13.843 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T07:53:13.950 INFO:teuthology.run:FAIL