2026-02-17T21:04:43.151 INFO:root:teuthology version: 0.0.1.dev31+g5f66ecfb3 2026-02-17T21:04:43.166 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T21:04:43.336 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889 branch: wip-yuri6-testing-2026-02-17-1732-squid description: rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_latest}} email: null first_in_suite: false flavor: default job_id: '53889' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri6-testing-2026-02-17-1732-squid ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - overall HEALTH_ - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-yuri6-testing-2026-02-17-1732-squid sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2987 sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-17-1732-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 targets: trial060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAEYQ3k/YawAFqDLdhxcMEz2lbT1ozEE2FaEYVxtNa50CLoN+S4GTQSB0kjdFrZDBuKz5KXHBQybE1+k4Y7jAE= tasks: - install: null - ceph: log-ignorelist: - \(POOL_APP_NOT_ENABLED\) - workunit: clients: all: - cephtool/test_kvstore_tool.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: 5f66ecfb34c0370410e78b3ee641753d19da653b timestamp: 2026-02-17_20:43:43 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T21:04:43.336 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa; will attempt to use it 2026-02-17T21:04:43.337 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks 2026-02-17T21:04:43.337 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T21:04:43.339 INFO:teuthology.task.internal:Checking packages... 2026-02-17T21:04:43.339 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1' 2026-02-17T21:04:43.339 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T21:04:43.339 INFO:teuthology.packaging:ref: None 2026-02-17T21:04:43.339 INFO:teuthology.packaging:tag: None 2026-02-17T21:04:43.339 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-17-1732-squid 2026-02-17T21:04:43.339 INFO:teuthology.packaging:sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:04:43.339 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri6-testing-2026-02-17-1732-squid 2026-02-17T21:04:43.470 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-679.gd855f53b 2026-02-17T21:04:43.471 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T21:04:43.483 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T21:04:43.483 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T21:04:43.502 INFO:teuthology.task.internal:Saving configuration 2026-02-17T21:04:43.514 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T21:04:43.527 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T21:04:45.154 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial060.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 21:01:19.184816', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAEYQ3k/YawAFqDLdhxcMEz2lbT1ozEE2FaEYVxtNa50CLoN+S4GTQSB0kjdFrZDBuKz5KXHBQybE1+k4Y7jAE='} 2026-02-17T21:04:45.154 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T21:04:45.165 INFO:teuthology.task.internal:roles: ubuntu@trial060.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T21:04:45.165 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T21:04:45.275 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7a70bd5fb920>, signals=[15]) 2026-02-17T21:04:45.276 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T21:04:45.289 INFO:teuthology.task.internal:Opening connections... 2026-02-17T21:04:45.289 DEBUG:teuthology.task.internal:connecting to ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:04:45.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:04:45.357 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T21:04:45.368 DEBUG:teuthology.orchestra.run.trial060:> uname -m 2026-02-17T21:04:45.433 INFO:teuthology.orchestra.run.trial060.stdout:x86_64 2026-02-17T21:04:45.433 DEBUG:teuthology.orchestra.run.trial060:> cat /etc/os-release 2026-02-17T21:04:45.490 INFO:teuthology.orchestra.run.trial060.stdout:NAME="CentOS Stream" 2026-02-17T21:04:45.490 INFO:teuthology.orchestra.run.trial060.stdout:VERSION="9" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:ID="centos" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:ID_LIKE="rhel fedora" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:VERSION_ID="9" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:PLATFORM_ID="platform:el9" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:ANSI_COLOR="0;31" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:LOGO="fedora-logo-icon" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:HOME_URL="https://centos.org/" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T21:04:45.491 INFO:teuthology.orchestra.run.trial060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T21:04:45.492 INFO:teuthology.lock.ops:Updating trial060.front.sepia.ceph.com on lock server 2026-02-17T21:04:45.584 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T21:04:45.597 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T21:04:45.608 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T21:04:45.608 DEBUG:teuthology.orchestra.run.trial060:> test '!' -e /home/ubuntu/cephtest 2026-02-17T21:04:45.623 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T21:04:45.634 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T21:04:45.634 DEBUG:teuthology.orchestra.run.trial060:> test -z $(ls -A /var/lib/ceph) 2026-02-17T21:04:45.680 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T21:04:45.786 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T21:04:45.806 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-17T21:04:45.806 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-17T21:04:45.806 DEBUG:teuthology.orchestra.run.trial060:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T21:04:45.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:04:45.822 DEBUG:teuthology.orchestra.run.trial060:> uname -r 2026-02-17T21:04:45.877 INFO:teuthology.orchestra.run.trial060.stdout:5.14.0-645.el9.x86_64 2026-02-17T21:04:45.878 INFO:teuthology.task.kernel:Running kernel on trial060: 5.14.0-645.el9.x86_64 2026-02-17T21:04:45.878 DEBUG:teuthology.orchestra.run.trial060:> sudo yum install -y kernel 2026-02-17T21:04:47.164 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.9 MB 00:00 2026-02-17T21:04:50.212 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T21:04:54.007 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 7.8 MB 00:01 2026-02-17T21:04:54.960 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 20 kB 00:00 2026-02-17T21:04:56.503 INFO:teuthology.orchestra.run.trial060.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2026-02-17T21:04:59.722 INFO:teuthology.orchestra.run.trial060.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-17T21:05:00.736 INFO:teuthology.orchestra.run.trial060.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T21:05:00.758 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:05:00.759 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:05:00.759 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-17T21:05:00.759 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:05:00.759 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:05:00.760 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:05:00.761 INFO:teuthology.orchestra.run.trial060.stdout:Install 4 Packages 2026-02-17T21:05:00.761 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:05:00.761 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 88 M 2026-02-17T21:05:00.761 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 135 M 2026-02-17T21:05:00.761 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:05:07.249 INFO:teuthology.orchestra.run.trial060.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 125 kB/s | 803 kB 00:06 2026-02-17T21:05:25.760 INFO:teuthology.orchestra.run.trial060.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 702 kB/s | 17 MB 00:24 2026-02-17T21:05:42.656 INFO:teuthology.orchestra.run.trial060.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 892 kB/s | 31 MB 00:35 2026-02-17T21:05:43.684 INFO:teuthology.orchestra.run.trial060.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 946 kB/s | 40 MB 00:42 2026-02-17T21:05:43.686 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:05:43.686 INFO:teuthology.orchestra.run.trial060.stdout:Total 2.1 MB/s | 88 MB 00:42 2026-02-17T21:05:43.875 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:05:44.079 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:05:44.079 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:05:44.427 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:05:44.427 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:05:45.521 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:05:45.806 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T21:05:45.825 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T21:05:47.621 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T21:05:47.650 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T21:05:50.857 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T21:05:50.863 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T21:05:53.969 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T21:06:24.479 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T21:06:24.500 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T21:06:24.564 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T21:06:24.564 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T21:06:24.565 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T21:06:24.565 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T21:06:24.618 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T21:06:24.618 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:06:24.618 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:06:24.618 INFO:teuthology.orchestra.run.trial060.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T21:06:24.618 INFO:teuthology.orchestra.run.trial060.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T21:06:24.619 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T21:06:24.619 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T21:06:24.619 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:06:24.619 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:06:24.674 DEBUG:teuthology.orchestra.run.trial060:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T21:06:24.701 INFO:teuthology.orchestra.run.trial060.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T21:06:24.702 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T21:06:24.702 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T21:06:24.702 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T21:06:24.702 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T21:06:24.703 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T21:06:24.703 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial060.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T21:06:24.703 DEBUG:teuthology.orchestra.run.trial060:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T21:06:24.765 INFO:teuthology.orchestra.run.trial060.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T21:06:24.767 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T21:06:24.767 DEBUG:teuthology.orchestra.run.trial060:> sudo rpm -qi grub2-tools 2026-02-17T21:06:24.837 INFO:teuthology.orchestra.run.trial060.stdout:Name : grub2-tools 2026-02-17T21:06:24.837 INFO:teuthology.orchestra.run.trial060.stdout:Epoch : 1 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Version : 2.06 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Release : 118.el9 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Architecture: x86_64 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Group : Unspecified 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Size : 8393909 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:License : GPLv3+ 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Packager : builder@centos.org 2026-02-17T21:06:24.838 INFO:teuthology.orchestra.run.trial060.stdout:Vendor : CentOS 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:Summary : Support tools for GRUB. 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:Description : 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:hardware devices. 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:06:24.839 INFO:teuthology.orchestra.run.trial060.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T21:06:24.840 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T21:06:24.840 INFO:teuthology.task.kernel:Updating grub on trial060 to boot 5.14.0-681.el9.x86_64 2026-02-17T21:06:24.841 DEBUG:teuthology.orchestra.run.trial060:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T21:06:24.904 INFO:teuthology.orchestra.run.trial060.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T21:06:24.906 DEBUG:teuthology.orchestra.run.trial060:> sudo command -v grubby && echo yes || echo no 2026-02-17T21:06:24.970 INFO:teuthology.orchestra.run.trial060.stdout:/sbin/grubby 2026-02-17T21:06:24.971 INFO:teuthology.orchestra.run.trial060.stdout:yes 2026-02-17T21:06:24.972 DEBUG:teuthology.orchestra.run.trial060:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T21:06:25.038 INFO:teuthology.orchestra.run.trial060.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T21:06:25.039 DEBUG:teuthology.orchestra.run.trial060:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T21:06:25.113 DEBUG:teuthology.orchestra.run.trial060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T21:06:25.324 INFO:teuthology.orchestra.run.trial060.stderr:Generating grub configuration file ... 2026-02-17T21:06:25.388 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T21:06:25.395 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T21:06:25.651 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T21:06:25.657 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T21:06:25.689 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T21:06:25.695 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T21:06:25.718 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T21:06:25.725 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T21:06:25.806 INFO:teuthology.orchestra.run.trial060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T21:06:25.814 INFO:teuthology.orchestra.run.trial060.stderr:done 2026-02-17T21:06:25.816 DEBUG:teuthology.orchestra.run.trial060:> test -d /sys/firmware/efi 2026-02-17T21:06:25.832 DEBUG:teuthology.orchestra.run.trial060:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T21:06:25.897 INFO:teuthology.orchestra.run.trial060.stdout:/boot/efi/EFI/BOOT 2026-02-17T21:06:25.897 INFO:teuthology.orchestra.run.trial060.stdout:/boot/efi/EFI/centos 2026-02-17T21:06:25.899 DEBUG:teuthology.orchestra.run.trial060:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T21:06:25.964 INFO:teuthology.orchestra.run.trial060.stdout:no 2026-02-17T21:06:25.965 DEBUG:teuthology.orchestra.run.trial060:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T21:06:26.028 INFO:teuthology.orchestra.run.trial060.stdout:yes 2026-02-17T21:06:26.029 DEBUG:teuthology.orchestra.run.trial060:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T21:06:26.093 DEBUG:teuthology.orchestra.run.trial060:> sudo shutdown -r now 2026-02-17T21:06:26.167 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-17T21:06:56.165 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T21:06:56.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-17T21:06:56.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:07:10.464 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.60 2026-02-17T21:07:19.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-17T21:07:19.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:07:39.146 ERROR:teuthology.orchestra.connection:Error authenticating with trial060.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2026-02-17T21:07:43.264 DEBUG:teuthology.orchestra.run.trial060:> true 2026-02-17T21:07:43.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-17T21:07:43.522 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-681.el9.x86_64"... 2026-02-17T21:07:43.523 DEBUG:teuthology.orchestra.run.trial060:> uname -r 2026-02-17T21:07:43.582 INFO:teuthology.orchestra.run.trial060.stdout:5.14.0-681.el9.x86_64 2026-02-17T21:07:43.583 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T21:07:43.583 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T21:07:43.583 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T21:07:44.584 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T21:07:44.585 DEBUG:teuthology.orchestra.run.trial060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T21:07:44.614 INFO:teuthology.orchestra.run.trial060.stdout:ttyS1 2026-02-17T21:07:44.651 DEBUG:teuthology.parallel:result is None 2026-02-17T21:07:44.652 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T21:07:44.662 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T21:07:44.662 DEBUG:teuthology.orchestra.run.trial060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T21:07:44.711 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T21:07:44.722 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T21:07:44.734 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T21:07:44.735 DEBUG:teuthology.orchestra.run.trial060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T21:07:44.777 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T21:07:44.789 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T21:07:44.789 DEBUG:teuthology.orchestra.run.trial060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T21:07:44.842 INFO:teuthology.orchestra.run.trial060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:07:44.850 INFO:teuthology.orchestra.run.trial060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:07:44.852 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T21:07:44.864 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T21:07:44.864 DEBUG:teuthology.orchestra.run.trial060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T21:07:44.922 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T21:07:44.938 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T21:07:44.938 DEBUG:teuthology.orchestra.run.trial060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T21:07:44.983 DEBUG:teuthology.orchestra.run.trial060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T21:07:45.148 DEBUG:teuthology.orchestra.run.trial060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T21:07:45.172 DEBUG:teuthology.orchestra.run.trial060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T21:07:45.325 DEBUG:teuthology.orchestra.run.trial060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T21:07:45.348 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:07:45.348 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T21:07:45.415 DEBUG:teuthology.orchestra.run.trial060:> sudo service rsyslog restart 2026-02-17T21:07:45.486 INFO:teuthology.orchestra.run.trial060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T21:07:45.661 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T21:07:45.674 INFO:teuthology.task.internal:Starting timer... 2026-02-17T21:07:45.674 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T21:07:45.695 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T21:07:45.715 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-17T21:07:45.716 DEBUG:teuthology.orchestra.run.trial060:> sudo service auditd rotate 2026-02-17T21:07:45.743 INFO:teuthology.orchestra.run.trial060.stdout:Rotating logs: 2026-02-17T21:07:45.745 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T21:07:45.745 DEBUG:teuthology.orchestra.run.trial060:> /usr/sbin/getenforce 2026-02-17T21:07:45.805 INFO:teuthology.orchestra.run.trial060.stdout:Permissive 2026-02-17T21:07:45.805 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial060.front.sepia.ceph.com': 'permissive'} 2026-02-17T21:07:45.806 DEBUG:teuthology.orchestra.run.trial060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T21:07:45.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:07:45.874 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T21:07:45.874 DEBUG:teuthology.orchestra.run.trial060:> sudo /usr/sbin/setenforce permissive 2026-02-17T21:07:45.941 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T21:07:45.957 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T21:07:45.971 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T21:07:45.973 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial060.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T21:08:56.827 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial060.front.sepia.ceph.com')] 2026-02-17T21:08:56.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-17T21:08:56.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:08:56.941 DEBUG:teuthology.orchestra.run.trial060:> true 2026-02-17T21:08:57.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-17T21:08:57.018 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T21:08:57.031 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T21:08:57.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T21:08:57.032 DEBUG:teuthology.orchestra.run.trial060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T21:08:57.085 INFO:teuthology.orchestra.run.trial060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T21:08:57.096 INFO:teuthology.orchestra.run.trial060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T21:08:57.117 INFO:teuthology.orchestra.run.trial060.stderr:sudo: ntpd: command not found 2026-02-17T21:08:57.127 INFO:teuthology.orchestra.run.trial060.stdout:506 Cannot talk to daemon 2026-02-17T21:08:57.139 INFO:teuthology.orchestra.run.trial060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T21:08:57.151 INFO:teuthology.orchestra.run.trial060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T21:08:57.198 INFO:teuthology.orchestra.run.trial060.stderr:bash: line 1: ntpq: command not found 2026-02-17T21:08:57.200 INFO:teuthology.orchestra.run.trial060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T21:08:57.200 INFO:teuthology.orchestra.run.trial060.stdout:=============================================================================== 2026-02-17T21:08:57.201 INFO:teuthology.run_tasks:Running task install... 2026-02-17T21:08:57.215 DEBUG:teuthology.task.install:project ceph 2026-02-17T21:08:57.215 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'}} 2026-02-17T21:08:57.215 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'} 2026-02-17T21:08:57.215 INFO:teuthology.task.install:Using flavor: default 2026-02-17T21:08:57.224 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T21:08:57.224 INFO:teuthology.task.install:extra packages: [] 2026-02-17T21:08:57.225 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1', 'tag': None, 'wait_for_package': False} 2026-02-17T21:08:57.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:08:57.387 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/centos/9/flavors/default/ 2026-02-17T21:08:57.387 INFO:teuthology.task.install.rpm:Package version is 19.2.3-679.gd855f53b 2026-02-17T21:08:57.521 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T21:08:57.521 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:08:57.522 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T21:08:57.547 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-17T21:08:57.547 DEBUG:teuthology.orchestra.run.trial060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T21:08:57.614 DEBUG:teuthology.orchestra.run.trial060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T21:08:57.687 DEBUG:teuthology.orchestra.run.trial060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T21:08:57.751 INFO:teuthology.orchestra.run.trial060.stdout:check_obsoletes = 1 2026-02-17T21:08:57.752 DEBUG:teuthology.orchestra.run.trial060:> sudo yum clean all 2026-02-17T21:08:57.899 INFO:teuthology.orchestra.run.trial060.stdout:41 files removed 2026-02-17T21:08:57.913 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-radosgw 2026-02-17T21:08:58.183 INFO:teuthology.orchestra.run.trial060.stdout:ceph packages for x86_64 678 kB/s | 86 kB 00:00 2026-02-17T21:08:58.316 INFO:teuthology.orchestra.run.trial060.stdout:ceph noarch packages 112 kB/s | 12 kB 00:00 2026-02-17T21:08:58.440 INFO:teuthology.orchestra.run.trial060.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-17T21:08:59.151 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-17T21:09:02.197 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T21:09:07.286 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 7.8 MB 00:02 2026-02-17T21:09:08.121 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - Extras packages 192 kB/s | 20 kB 00:00 2026-02-17T21:09:09.287 INFO:teuthology.orchestra.run.trial060.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2026-02-17T21:09:12.504 INFO:teuthology.orchestra.run.trial060.stdout:lab-extras 1.8 MB/s | 50 kB 00:00 2026-02-17T21:09:13.721 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repository Size 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout: ceph-radosgw x86_64 2:19.2.3-679.gd855f53b.el9 ceph 11 M 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout:Upgrading: 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout: librados2 x86_64 2:19.2.3-679.gd855f53b.el9 ceph 3.4 M 2026-02-17T21:09:13.723 INFO:teuthology.orchestra.run.trial060.stdout: librbd1 x86_64 2:19.2.3-679.gd855f53b.el9 ceph 3.2 M 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: ceph-base x86_64 2:19.2.3-679.gd855f53b.el9 ceph 5.5 M 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: ceph-common x86_64 2:19.2.3-679.gd855f53b.el9 ceph 22 M 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: ceph-selinux x86_64 2:19.2.3-679.gd855f53b.el9 ceph 25 k 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs2 x86_64 2:19.2.3-679.gd855f53b.el9 ceph 1.0 M 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T21:09:13.724 INFO:teuthology.orchestra.run.trial060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: libradosstriper1 x86_64 2:19.2.3-679.gd855f53b.el9 ceph 504 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: librgw2 x86_64 2:19.2.3-679.gd855f53b.el9 ceph 5.4 M 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-argparse x86_64 2:19.2.3-679.gd855f53b.el9 ceph 45 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-common x86_64 2:19.2.3-679.gd855f53b.el9 ceph 142 k 2026-02-17T21:09:13.725 INFO:teuthology.orchestra.run.trial060.stdout: python3-cephfs x86_64 2:19.2.3-679.gd855f53b.el9 ceph 165 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: python3-rados x86_64 2:19.2.3-679.gd855f53b.el9 ceph 322 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: python3-rbd x86_64 2:19.2.3-679.gd855f53b.el9 ceph 303 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: python3-rgw x86_64 2:19.2.3-679.gd855f53b.el9 ceph 100 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout:Installing weak dependencies: 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:13.726 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:09:13.727 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:09:13.727 INFO:teuthology.orchestra.run.trial060.stdout:Install 33 Packages 2026-02-17T21:09:13.727 INFO:teuthology.orchestra.run.trial060.stdout:Upgrade 2 Packages 2026-02-17T21:09:13.727 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:13.727 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 63 M 2026-02-17T21:09:13.728 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:09:15.367 INFO:teuthology.orchestra.run.trial060.stdout:(1/35): ceph-base-19.2.3-679.gd855f53b.el9.x86_ 3.8 MB/s | 5.5 MB 00:01 2026-02-17T21:09:15.423 INFO:teuthology.orchestra.run.trial060.stdout:(2/35): ceph-selinux-19.2.3-679.gd855f53b.el9.x 445 kB/s | 25 kB 00:00 2026-02-17T21:09:15.673 INFO:teuthology.orchestra.run.trial060.stdout:(3/35): libcephfs2-19.2.3-679.gd855f53b.el9.x86 3.9 MB/s | 1.0 MB 00:00 2026-02-17T21:09:15.807 INFO:teuthology.orchestra.run.trial060.stdout:(4/35): libradosstriper1-19.2.3-679.gd855f53b.e 3.7 MB/s | 504 kB 00:00 2026-02-17T21:09:15.881 INFO:teuthology.orchestra.run.trial060.stdout:(5/35): ceph-radosgw-19.2.3-679.gd855f53b.el9.x 5.5 MB/s | 11 MB 00:01 2026-02-17T21:09:15.937 INFO:teuthology.orchestra.run.trial060.stdout:(6/35): python3-ceph-argparse-19.2.3-679.gd855f 806 kB/s | 45 kB 00:00 2026-02-17T21:09:15.999 INFO:teuthology.orchestra.run.trial060.stdout:(7/35): python3-ceph-common-19.2.3-679.gd855f53 2.2 MB/s | 142 kB 00:00 2026-02-17T21:09:16.064 INFO:teuthology.orchestra.run.trial060.stdout:(8/35): python3-cephfs-19.2.3-679.gd855f53b.el9 2.5 MB/s | 165 kB 00:00 2026-02-17T21:09:16.133 INFO:teuthology.orchestra.run.trial060.stdout:(9/35): python3-rados-19.2.3-679.gd855f53b.el9. 4.6 MB/s | 322 kB 00:00 2026-02-17T21:09:16.202 INFO:teuthology.orchestra.run.trial060.stdout:(10/35): python3-rbd-19.2.3-679.gd855f53b.el9.x 4.3 MB/s | 303 kB 00:00 2026-02-17T21:09:16.267 INFO:teuthology.orchestra.run.trial060.stdout:(11/35): python3-rgw-19.2.3-679.gd855f53b.el9.x 1.5 MB/s | 100 kB 00:00 2026-02-17T21:09:16.297 INFO:teuthology.orchestra.run.trial060.stdout:(12/35): mailcap-2.1.49-5.el9.noarch.rpm 1.1 MB/s | 33 kB 00:00 2026-02-17T21:09:16.339 INFO:teuthology.orchestra.run.trial060.stdout:(13/35): nvme-cli-2.13-1.el9.x86_64.rpm 26 MB/s | 1.1 MB 00:00 2026-02-17T21:09:16.349 INFO:teuthology.orchestra.run.trial060.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 9.9 MB/s | 93 kB 00:00 2026-02-17T21:09:16.398 INFO:teuthology.orchestra.run.trial060.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 2.1 MB/s | 104 kB 00:00 2026-02-17T21:09:16.415 INFO:teuthology.orchestra.run.trial060.stdout:(16/35): libnbd-1.20.3-4.el9.x86_64.rpm 9.5 MB/s | 164 kB 00:00 2026-02-17T21:09:16.426 INFO:teuthology.orchestra.run.trial060.stdout:(17/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 15 MB/s | 160 kB 00:00 2026-02-17T21:09:16.434 INFO:teuthology.orchestra.run.trial060.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 5.3 MB/s | 45 kB 00:00 2026-02-17T21:09:16.455 INFO:teuthology.orchestra.run.trial060.stdout:(19/35): librdkafka-1.6.1-102.el9.x86_64.rpm 32 MB/s | 662 kB 00:00 2026-02-17T21:09:16.465 INFO:teuthology.orchestra.run.trial060.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 28 MB/s | 292 kB 00:00 2026-02-17T21:09:16.475 INFO:teuthology.orchestra.run.trial060.stdout:(21/35): qatlib-25.08.0-2.el9.x86_64.rpm 23 MB/s | 240 kB 00:00 2026-02-17T21:09:16.483 INFO:teuthology.orchestra.run.trial060.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 4.4 MB/s | 37 kB 00:00 2026-02-17T21:09:16.492 INFO:teuthology.orchestra.run.trial060.stdout:(23/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 7.5 MB/s | 66 kB 00:00 2026-02-17T21:09:16.860 INFO:teuthology.orchestra.run.trial060.stdout:(24/35): librgw2-19.2.3-679.gd855f53b.el9.x86_6 5.1 MB/s | 5.4 MB 00:01 2026-02-17T21:09:17.000 INFO:teuthology.orchestra.run.trial060.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 607 kB/s | 308 kB 00:00 2026-02-17T21:09:17.071 INFO:teuthology.orchestra.run.trial060.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 349 kB/s | 25 kB 00:00 2026-02-17T21:09:17.145 INFO:teuthology.orchestra.run.trial060.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 655 kB/s | 49 kB 00:00 2026-02-17T21:09:17.222 INFO:teuthology.orchestra.run.trial060.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 873 kB/s | 67 kB 00:00 2026-02-17T21:09:17.468 INFO:teuthology.orchestra.run.trial060.stdout:(29/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2026-02-17T21:09:17.546 INFO:teuthology.orchestra.run.trial060.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 2.4 MB/s | 191 kB 00:00 2026-02-17T21:09:17.684 INFO:teuthology.orchestra.run.trial060.stdout:(31/35): libarrow-9.0.0-15.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2026-02-17T21:09:17.730 INFO:teuthology.orchestra.run.trial060.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2026-02-17T21:09:18.087 INFO:teuthology.orchestra.run.trial060.stdout:(33/35): ceph-common-19.2.3-679.gd855f53b.el9.x 5.2 MB/s | 22 MB 00:04 2026-02-17T21:09:18.292 INFO:teuthology.orchestra.run.trial060.stdout:(34/35): librbd1-19.2.3-679.gd855f53b.el9.x86_6 5.6 MB/s | 3.2 MB 00:00 2026-02-17T21:09:18.413 INFO:teuthology.orchestra.run.trial060.stdout:(35/35): librados2-19.2.3-679.gd855f53b.el9.x86 4.7 MB/s | 3.4 MB 00:00 2026-02-17T21:09:18.415 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:09:18.415 INFO:teuthology.orchestra.run.trial060.stdout:Total 13 MB/s | 63 MB 00:04 2026-02-17T21:09:18.516 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:09:18.535 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:09:18.535 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:09:18.787 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:09:18.787 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:09:19.083 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:09:19.091 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-17T21:09:19.094 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-17T21:09:19.106 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-17T21:09:19.257 INFO:teuthology.orchestra.run.trial060.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-17T21:09:19.262 INFO:teuthology.orchestra.run.trial060.stdout: Upgrading : librados2-2:19.2.3-679.gd855f53b.el9.x86_64 5/37 2026-02-17T21:09:19.291 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librados2-2:19.2.3-679.gd855f53b.el9.x86_64 5/37 2026-02-17T21:09:19.335 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rados-2:19.2.3-679.gd855f53b.el9.x86_64 6/37 2026-02-17T21:09:19.336 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephfs2-2:19.2.3-679.gd855f53b.el9.x86_64 7/37 2026-02-17T21:09:19.349 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephfs2-2:19.2.3-679.gd855f53b.el9.x86_64 7/37 2026-02-17T21:09:19.353 INFO:teuthology.orchestra.run.trial060.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-17T21:09:19.363 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-ceph-argparse-2:19.2.3-679.gd855f53b.el9.x 9/37 2026-02-17T21:09:19.386 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cephfs-2:19.2.3-679.gd855f53b.el9.x86_64 10/37 2026-02-17T21:09:19.387 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libradosstriper1-2:19.2.3-679.gd855f53b.el9.x86_64 11/37 2026-02-17T21:09:19.400 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libradosstriper1-2:19.2.3-679.gd855f53b.el9.x86_64 11/37 2026-02-17T21:09:19.436 INFO:teuthology.orchestra.run.trial060.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-17T21:09:19.440 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-17T21:09:19.446 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-17T21:09:19.449 INFO:teuthology.orchestra.run.trial060.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-17T21:09:19.452 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-17T21:09:19.457 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-17T21:09:19.598 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T21:09:19.599 INFO:teuthology.orchestra.run.trial060.stdout: Upgrading : librbd1-2:19.2.3-679.gd855f53b.el9.x86_64 19/37 2026-02-17T21:09:19.625 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librbd1-2:19.2.3-679.gd855f53b.el9.x86_64 19/37 2026-02-17T21:09:19.630 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rbd-2:19.2.3-679.gd855f53b.el9.x86_64 20/37 2026-02-17T21:09:19.636 INFO:teuthology.orchestra.run.trial060.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-17T21:09:19.861 INFO:teuthology.orchestra.run.trial060.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-17T21:09:19.862 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librgw2-2:19.2.3-679.gd855f53b.el9.x86_64 23/37 2026-02-17T21:09:19.878 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librgw2-2:19.2.3-679.gd855f53b.el9.x86_64 23/37 2026-02-17T21:09:19.882 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rgw-2:19.2.3-679.gd855f53b.el9.x86_64 24/37 2026-02-17T21:09:19.886 INFO:teuthology.orchestra.run.trial060.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-17T21:09:19.889 INFO:teuthology.orchestra.run.trial060.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-17T21:09:19.889 INFO:teuthology.orchestra.run.trial060.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T21:09:19.907 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-17T21:09:20.020 INFO:teuthology.orchestra.run.trial060.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-17T21:09:20.023 INFO:teuthology.orchestra.run.trial060.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T21:09:20.308 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 29/37 2026-02-17T21:09:20.308 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T21:09:20.308 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:20.343 INFO:teuthology.orchestra.run.trial060.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-17T21:09:20.345 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-ceph-common-2:19.2.3-679.gd855f53b.el9.x86 31/37 2026-02-17T21:09:21.325 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-common-2:19.2.3-679.gd855f53b.el9.x86_64 32/37 2026-02-17T21:09:21.327 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-common-2:19.2.3-679.gd855f53b.el9.x86_64 32/37 2026-02-17T21:09:21.608 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-common-2:19.2.3-679.gd855f53b.el9.x86_64 32/37 2026-02-17T21:09:21.611 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-base-2:19.2.3-679.gd855f53b.el9.x86_64 33/37 2026-02-17T21:09:21.651 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-base-2:19.2.3-679.gd855f53b.el9.x86_64 33/37 2026-02-17T21:09:21.652 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T21:09:21.652 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T21:09:21.652 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:21.655 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-selinux-2:19.2.3-679.gd855f53b.el9.x86_64 34/37 2026-02-17T21:09:26.169 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-selinux-2:19.2.3-679.gd855f53b.el9.x86_64 34/37 2026-02-17T21:09:26.169 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /sys 2026-02-17T21:09:26.169 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /proc 2026-02-17T21:09:26.169 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /mnt 2026-02-17T21:09:26.169 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /var/tmp 2026-02-17T21:09:26.169 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /home 2026-02-17T21:09:26.170 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /root 2026-02-17T21:09:26.170 INFO:teuthology.orchestra.run.trial060.stdout:skipping the directory /tmp 2026-02-17T21:09:26.170 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:26.619 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-radosgw-2:19.2.3-679.gd855f53b.el9.x86_64 35/37 2026-02-17T21:09:26.642 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-679.gd855f53b.el9.x86_64 35/37 2026-02-17T21:09:26.642 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:09:26.642 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T21:09:26.642 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T21:09:26.642 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T21:09:26.642 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:26.644 INFO:teuthology.orchestra.run.trial060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T21:09:26.655 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-17T21:09:26.655 INFO:teuthology.orchestra.run.trial060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T21:09:26.711 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T21:09:26.711 INFO:teuthology.orchestra.run.trial060.stdout:Creating group 'qat' with GID 992. 2026-02-17T21:09:26.711 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:26.985 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-base-2:19.2.3-679.gd855f53b.el9.x86_64 1/37 2026-02-17T21:09:26.985 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-common-2:19.2.3-679.gd855f53b.el9.x86_64 2/37 2026-02-17T21:09:26.985 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-radosgw-2:19.2.3-679.gd855f53b.el9.x86_64 3/37 2026-02-17T21:09:26.985 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-selinux-2:19.2.3-679.gd855f53b.el9.x86_64 4/37 2026-02-17T21:09:26.985 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephfs2-2:19.2.3-679.gd855f53b.el9.x86_64 5/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libradosstriper1-2:19.2.3-679.gd855f53b.el9.x86_64 6/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librgw2-2:19.2.3-679.gd855f53b.el9.x86_64 7/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-ceph-argparse-2:19.2.3-679.gd855f53b.el9.x 8/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-ceph-common-2:19.2.3-679.gd855f53b.el9.x86 9/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cephfs-2:19.2.3-679.gd855f53b.el9.x86_64 10/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rados-2:19.2.3-679.gd855f53b.el9.x86_64 11/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rbd-2:19.2.3-679.gd855f53b.el9.x86_64 12/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rgw-2:19.2.3-679.gd855f53b.el9.x86_64 13/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 15/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-17T21:09:26.986 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-17T21:09:26.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-17T21:09:26.988 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-17T21:09:26.988 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-17T21:09:26.988 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librados2-2:19.2.3-679.gd855f53b.el9.x86_64 34/37 2026-02-17T21:09:26.988 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-17T21:09:26.988 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librbd1-2:19.2.3-679.gd855f53b.el9.x86_64 36/37 2026-02-17T21:09:27.041 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout:Upgraded: 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: librados2-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: librbd1-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: ceph-base-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: ceph-common-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: ceph-radosgw-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: ceph-selinux-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.042 INFO:teuthology.orchestra.run.trial060.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs2-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libradosstriper1-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: librgw2-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T21:09:27.043 INFO:teuthology.orchestra.run.trial060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-argparse-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: python3-ceph-common-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: python3-cephfs-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: python3-rados-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: python3-rbd-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: python3-rgw-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T21:09:27.044 INFO:teuthology.orchestra.run.trial060.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T21:09:27.045 INFO:teuthology.orchestra.run.trial060.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T21:09:27.045 INFO:teuthology.orchestra.run.trial060.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T21:09:27.045 INFO:teuthology.orchestra.run.trial060.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T21:09:27.045 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:27.045 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:09:27.083 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-test 2026-02-17T21:09:27.399 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:15 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:09:27.481 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:09:27.482 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:09:27.482 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repository Size 2026-02-17T21:09:27.482 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:09:27.482 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:09:27.482 INFO:teuthology.orchestra.run.trial060.stdout: ceph-test x86_64 2:19.2.3-679.gd855f53b.el9 ceph 50 M 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout: libcephsqlite x86_64 2:19.2.3-679.gd855f53b.el9 ceph 163 k 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout:Install 5 Packages 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:27.483 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 50 M 2026-02-17T21:09:27.484 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 212 M 2026-02-17T21:09:27.484 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:09:27.623 INFO:teuthology.orchestra.run.trial060.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 3.0 MB/s | 233 kB 00:00 2026-02-17T21:09:27.665 INFO:teuthology.orchestra.run.trial060.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 7.0 MB/s | 303 kB 00:00 2026-02-17T21:09:27.697 INFO:teuthology.orchestra.run.trial060.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.0 MB/s | 64 kB 00:00 2026-02-17T21:09:28.775 INFO:teuthology.orchestra.run.trial060.stdout:(4/5): libcephsqlite-19.2.3-679.gd855f53b.el9.x 133 kB/s | 163 kB 00:01 2026-02-17T21:09:38.812 INFO:teuthology.orchestra.run.trial060.stdout:(5/5): ceph-test-19.2.3-679.gd855f53b.el9.x86_6 4.4 MB/s | 50 MB 00:11 2026-02-17T21:09:38.814 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:09:38.814 INFO:teuthology.orchestra.run.trial060.stdout:Total 4.4 MB/s | 50 MB 00:11 2026-02-17T21:09:38.840 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:09:38.849 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:09:38.849 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:09:39.036 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:09:39.037 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:09:39.243 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:09:39.249 INFO:teuthology.orchestra.run.trial060.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-17T21:09:39.253 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-17T21:09:39.261 INFO:teuthology.orchestra.run.trial060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-17T21:09:39.262 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephsqlite-2:19.2.3-679.gd855f53b.el9.x86_64 4/5 2026-02-17T21:09:41.557 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephsqlite-2:19.2.3-679.gd855f53b.el9.x86_64 4/5 2026-02-17T21:09:41.575 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-test-2:19.2.3-679.gd855f53b.el9.x86_64 5/5 2026-02-17T21:09:41.803 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-test-2:19.2.3-679.gd855f53b.el9.x86_64 5/5 2026-02-17T21:09:41.803 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-test-2:19.2.3-679.gd855f53b.el9.x86_64 1/5 2026-02-17T21:09:41.803 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephsqlite-2:19.2.3-679.gd855f53b.el9.x86_64 2/5 2026-02-17T21:09:41.803 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-17T21:09:41.803 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-17T21:09:41.855 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-17T21:09:41.855 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout: ceph-test-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout: libcephsqlite-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:41.856 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:09:41.890 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph 2026-02-17T21:09:42.196 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:09:42.280 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:09:42.283 INFO:teuthology.orchestra.run.trial060.stdout:====================================================================================== 2026-02-17T21:09:42.283 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repository Size 2026-02-17T21:09:42.283 INFO:teuthology.orchestra.run.trial060.stdout:====================================================================================== 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph x86_64 2:19.2.3-679.gd855f53b.el9 ceph 6.5 k 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-grafana-dashboards noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 31 k 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mds x86_64 2:19.2.3-679.gd855f53b.el9 ceph 2.4 M 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr x86_64 2:19.2.3-679.gd855f53b.el9 ceph 1.1 M 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-modules-core noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 253 k 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mon x86_64 2:19.2.3-679.gd855f53b.el9 ceph 4.7 M 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd x86_64 2:19.2.3-679.gd855f53b.el9 ceph 17 M 2026-02-17T21:09:42.284 INFO:teuthology.orchestra.run.trial060.stdout: ceph-prometheus-alerts noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 17 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: cephadm noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 769 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T21:09:42.285 INFO:teuthology.orchestra.run.trial060.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T21:09:42.286 INFO:teuthology.orchestra.run.trial060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T21:09:42.287 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T21:09:42.288 INFO:teuthology.orchestra.run.trial060.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T21:09:42.289 INFO:teuthology.orchestra.run.trial060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T21:09:42.290 INFO:teuthology.orchestra.run.trial060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T21:09:42.291 INFO:teuthology.orchestra.run.trial060.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout:Installing weak dependencies: 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-cephadm noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 150 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-dashboard noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 3.8 M 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 7.4 M 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 22 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-rook noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 49 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: ceph-volume noarch 2:19.2.3-679.gd855f53b.el9 ceph-noarch 299 k 2026-02-17T21:09:42.292 INFO:teuthology.orchestra.run.trial060.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T21:09:42.293 INFO:teuthology.orchestra.run.trial060.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T21:09:42.293 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:42.293 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:09:42.293 INFO:teuthology.orchestra.run.trial060.stdout:====================================================================================== 2026-02-17T21:09:42.293 INFO:teuthology.orchestra.run.trial060.stdout:Install 99 Packages 2026-02-17T21:09:42.293 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:09:42.294 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 95 M 2026-02-17T21:09:42.294 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 520 M 2026-02-17T21:09:42.294 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:09:43.221 INFO:teuthology.orchestra.run.trial060.stdout:(1/99): ceph-19.2.3-679.gd855f53b.el9.x86_64.rp 114 kB/s | 6.5 kB 00:00 2026-02-17T21:09:43.313 INFO:teuthology.orchestra.run.trial060.stdout:(2/99): ceph-mgr-19.2.3-679.gd855f53b.el9.x86_6 7.2 MB/s | 1.1 MB 00:00 2026-02-17T21:09:43.339 INFO:teuthology.orchestra.run.trial060.stdout:(3/99): ceph-mds-19.2.3-679.gd855f53b.el9.x86_6 14 MB/s | 2.4 MB 00:00 2026-02-17T21:09:43.372 INFO:teuthology.orchestra.run.trial060.stdout:(4/99): ceph-grafana-dashboards-19.2.3-679.gd85 943 kB/s | 31 kB 00:00 2026-02-17T21:09:43.410 INFO:teuthology.orchestra.run.trial060.stdout:(5/99): ceph-mgr-cephadm-19.2.3-679.gd855f53b.e 3.9 MB/s | 150 kB 00:00 2026-02-17T21:09:43.440 INFO:teuthology.orchestra.run.trial060.stdout:(6/99): ceph-mon-19.2.3-679.gd855f53b.el9.x86_6 22 MB/s | 4.7 MB 00:00 2026-02-17T21:09:43.583 INFO:teuthology.orchestra.run.trial060.stdout:(7/99): ceph-mgr-dashboard-19.2.3-679.gd855f53b 22 MB/s | 3.8 MB 00:00 2026-02-17T21:09:43.622 INFO:teuthology.orchestra.run.trial060.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-679.gd855f53b 572 kB/s | 22 kB 00:00 2026-02-17T21:09:43.670 INFO:teuthology.orchestra.run.trial060.stdout:(9/99): ceph-mgr-modules-core-19.2.3-679.gd855f 5.1 MB/s | 253 kB 00:00 2026-02-17T21:09:43.717 INFO:teuthology.orchestra.run.trial060.stdout:(10/99): ceph-mgr-rook-19.2.3-679.gd855f53b.el9 1.0 MB/s | 49 kB 00:00 2026-02-17T21:09:43.748 INFO:teuthology.orchestra.run.trial060.stdout:(11/99): ceph-mgr-diskprediction-local-19.2.3-6 24 MB/s | 7.4 MB 00:00 2026-02-17T21:09:43.748 INFO:teuthology.orchestra.run.trial060.stdout:(12/99): ceph-prometheus-alerts-19.2.3-679.gd85 526 kB/s | 17 kB 00:00 2026-02-17T21:09:43.775 INFO:teuthology.orchestra.run.trial060.stdout:(13/99): ceph-volume-19.2.3-679.gd855f53b.el9.n 11 MB/s | 299 kB 00:00 2026-02-17T21:09:43.789 INFO:teuthology.orchestra.run.trial060.stdout:(14/99): cephadm-19.2.3-679.gd855f53b.el9.noarc 18 MB/s | 769 kB 00:00 2026-02-17T21:09:43.853 INFO:teuthology.orchestra.run.trial060.stdout:(15/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 638 kB/s | 40 kB 00:00 2026-02-17T21:09:43.876 INFO:teuthology.orchestra.run.trial060.stdout:(16/99): libconfig-1.7.2-9.el9.x86_64.rpm 3.1 MB/s | 72 kB 00:00 2026-02-17T21:09:43.886 INFO:teuthology.orchestra.run.trial060.stdout:(17/99): cryptsetup-2.8.1-2.el9.x86_64.rpm 3.1 MB/s | 350 kB 00:00 2026-02-17T21:09:43.925 INFO:teuthology.orchestra.run.trial060.stdout:(18/99): ceph-osd-19.2.3-679.gd855f53b.el9.x86_ 28 MB/s | 17 MB 00:00 2026-02-17T21:09:43.925 INFO:teuthology.orchestra.run.trial060.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 4.6 MB/s | 184 kB 00:00 2026-02-17T21:09:43.947 INFO:teuthology.orchestra.run.trial060.stdout:(20/99): libgfortran-11.5.0-14.el9.x86_64.rpm 11 MB/s | 794 kB 00:00 2026-02-17T21:09:43.971 INFO:teuthology.orchestra.run.trial060.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 4.4 MB/s | 106 kB 00:00 2026-02-17T21:09:43.994 INFO:teuthology.orchestra.run.trial060.stdout:(22/99): python3-cryptography-36.0.1-5.el9.x86_ 18 MB/s | 1.2 MB 00:00 2026-02-17T21:09:43.995 INFO:teuthology.orchestra.run.trial060.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 5.5 MB/s | 135 kB 00:00 2026-02-17T21:09:44.011 INFO:teuthology.orchestra.run.trial060.stdout:(24/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 253 kB 00:00 2026-02-17T21:09:44.018 INFO:teuthology.orchestra.run.trial060.stdout:(25/99): python3-pyparsing-2.4.7-9.el9.noarch.r 6.1 MB/s | 150 kB 00:00 2026-02-17T21:09:44.019 INFO:teuthology.orchestra.run.trial060.stdout:(26/99): python3-requests-2.25.1-10.el9.noarch. 5.2 MB/s | 126 kB 00:00 2026-02-17T21:09:44.037 INFO:teuthology.orchestra.run.trial060.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 8.2 MB/s | 218 kB 00:00 2026-02-17T21:09:44.041 INFO:teuthology.orchestra.run.trial060.stdout:(28/99): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-17T21:09:44.043 INFO:teuthology.orchestra.run.trial060.stdout:(29/99): unzip-6.0-59.el9.x86_64.rpm 7.1 MB/s | 182 kB 00:00 2026-02-17T21:09:44.536 INFO:teuthology.orchestra.run.trial060.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 30 kB/s | 15 kB 00:00 2026-02-17T21:09:44.541 INFO:teuthology.orchestra.run.trial060.stdout:(31/99): flexiblas-3.0.4-9.el9.x86_64.rpm 59 kB/s | 30 kB 00:00 2026-02-17T21:09:44.872 INFO:teuthology.orchestra.run.trial060.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 735 kB/s | 246 kB 00:00 2026-02-17T21:09:44.875 INFO:teuthology.orchestra.run.trial060.stdout:(33/99): lua-5.4.4-4.el9.x86_64.rpm 565 kB/s | 188 kB 00:00 2026-02-17T21:09:44.972 INFO:teuthology.orchestra.run.trial060.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 422 kB/s | 42 kB 00:00 2026-02-17T21:09:46.173 INFO:teuthology.orchestra.run.trial060.stdout:(35/99): protobuf-3.14.0-17.el9.x86_64.rpm 856 kB/s | 1.0 MB 00:01 2026-02-17T21:09:48.082 INFO:teuthology.orchestra.run.trial060.stdout:(36/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 758 kB/s | 3.0 MB 00:04 2026-02-17T21:09:48.418 INFO:teuthology.orchestra.run.trial060.stdout:(37/99): python3-devel-3.9.25-2.el9.x86_64.rpm 727 kB/s | 244 kB 00:00 2026-02-17T21:09:48.754 INFO:teuthology.orchestra.run.trial060.stdout:(38/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 740 kB/s | 249 kB 00:00 2026-02-17T21:09:49.012 INFO:teuthology.orchestra.run.trial060.stdout:(39/99): python3-libstoragemgmt-1.10.1-1.el9.x8 688 kB/s | 177 kB 00:00 2026-02-17T21:09:50.449 INFO:teuthology.orchestra.run.trial060.stdout:(40/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 871 kB/s | 1.2 MB 00:01 2026-02-17T21:09:50.707 INFO:teuthology.orchestra.run.trial060.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 668 kB/s | 172 kB 00:00 2026-02-17T21:09:50.807 INFO:teuthology.orchestra.run.trial060.stdout:(42/99): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2026-02-17T21:09:50.933 INFO:teuthology.orchestra.run.trial060.stdout:(43/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 893 kB/s | 5.3 MB 00:06 2026-02-17T21:09:51.503 INFO:teuthology.orchestra.run.trial060.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 776 kB/s | 442 kB 00:00 2026-02-17T21:09:51.682 INFO:teuthology.orchestra.run.trial060.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 430 kB/s | 77 kB 00:00 2026-02-17T21:09:52.017 INFO:teuthology.orchestra.run.trial060.stdout:(46/99): python3-protobuf-3.14.0-17.el9.noarch. 799 kB/s | 267 kB 00:00 2026-02-17T21:09:52.274 INFO:teuthology.orchestra.run.trial060.stdout:(47/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 618 kB/s | 159 kB 00:00 2026-02-17T21:09:52.609 INFO:teuthology.orchestra.run.trial060.stdout:(48/99): python3-pyasn1-modules-0.4.8-6.el9.noa 832 kB/s | 279 kB 00:00 2026-02-17T21:09:52.710 INFO:teuthology.orchestra.run.trial060.stdout:(49/99): python3-requests-oauthlib-1.3.0-12.el9 534 kB/s | 54 kB 00:00 2026-02-17T21:09:53.044 INFO:teuthology.orchestra.run.trial060.stdout:(50/99): python3-babel-2.9.1-2.el9.noarch.rpm 889 kB/s | 6.0 MB 00:06 2026-02-17T21:09:53.144 INFO:teuthology.orchestra.run.trial060.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2026-02-17T21:09:53.401 INFO:teuthology.orchestra.run.trial060.stdout:(52/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 734 kB/s | 189 kB 00:00 2026-02-17T21:09:53.580 INFO:teuthology.orchestra.run.trial060.stdout:(53/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 505 kB/s | 90 kB 00:00 2026-02-17T21:09:53.605 INFO:teuthology.orchestra.run.trial060.stdout:(54/99): lua-devel-5.4.4-4.el9.x86_64.rpm 908 kB/s | 22 kB 00:00 2026-02-17T21:09:53.646 INFO:teuthology.orchestra.run.trial060.stdout:(55/99): protobuf-compiler-3.14.0-17.el9.x86_64 20 MB/s | 862 kB 00:00 2026-02-17T21:09:53.940 INFO:teuthology.orchestra.run.trial060.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.8 MB/s | 551 kB 00:00 2026-02-17T21:09:53.968 INFO:teuthology.orchestra.run.trial060.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 676 kB/s | 19 kB 00:00 2026-02-17T21:09:53.999 INFO:teuthology.orchestra.run.trial060.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 4.9 MB/s | 151 kB 00:00 2026-02-17T21:09:54.031 INFO:teuthology.orchestra.run.trial060.stdout:(59/99): python3-asyncssh-2.13.2-5.el9.noarch.r 16 MB/s | 548 kB 00:00 2026-02-17T21:09:54.061 INFO:teuthology.orchestra.run.trial060.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 1.0 MB/s | 29 kB 00:00 2026-02-17T21:09:54.090 INFO:teuthology.orchestra.run.trial060.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 2.0 MB/s | 60 kB 00:00 2026-02-17T21:09:54.119 INFO:teuthology.orchestra.run.trial060.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.5 MB/s | 43 kB 00:00 2026-02-17T21:09:54.148 INFO:teuthology.orchestra.run.trial060.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 1.1 MB/s | 32 kB 00:00 2026-02-17T21:09:54.177 INFO:teuthology.orchestra.run.trial060.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 492 kB/s | 14 kB 00:00 2026-02-17T21:09:54.207 INFO:teuthology.orchestra.run.trial060.stdout:(65/99): python3-cheroot-10.0.1-4.el9.noarch.rp 5.5 MB/s | 173 kB 00:00 2026-02-17T21:09:54.243 INFO:teuthology.orchestra.run.trial060.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 9.9 MB/s | 358 kB 00:00 2026-02-17T21:09:54.274 INFO:teuthology.orchestra.run.trial060.stdout:(67/99): python3-google-auth-2.45.0-1.el9.noarc 8.0 MB/s | 254 kB 00:00 2026-02-17T21:09:54.367 INFO:teuthology.orchestra.run.trial060.stdout:(68/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 22 MB/s | 2.0 MB 00:00 2026-02-17T21:09:54.398 INFO:teuthology.orchestra.run.trial060.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 4.6 MB/s | 144 kB 00:00 2026-02-17T21:09:54.429 INFO:teuthology.orchestra.run.trial060.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 4.4 MB/s | 139 kB 00:00 2026-02-17T21:09:54.458 INFO:teuthology.orchestra.run.trial060.stdout:(71/99): python3-isodate-0.6.1-3.el9.noarch.rpm 1.8 MB/s | 56 kB 00:00 2026-02-17T21:09:54.487 INFO:teuthology.orchestra.run.trial060.stdout:(72/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 369 kB/s | 11 kB 00:00 2026-02-17T21:09:54.516 INFO:teuthology.orchestra.run.trial060.stdout:(73/99): python3-jaraco-classes-3.2.1-5.el9.noa 621 kB/s | 18 kB 00:00 2026-02-17T21:09:54.545 INFO:teuthology.orchestra.run.trial060.stdout:(74/99): python3-jaraco-collections-3.0.0-8.el9 811 kB/s | 23 kB 00:00 2026-02-17T21:09:54.573 INFO:teuthology.orchestra.run.trial060.stdout:(75/99): python3-jaraco-context-6.0.1-3.el9.noa 689 kB/s | 20 kB 00:00 2026-02-17T21:09:54.602 INFO:teuthology.orchestra.run.trial060.stdout:(76/99): python3-jaraco-functools-3.5.0-2.el9.n 680 kB/s | 19 kB 00:00 2026-02-17T21:09:54.631 INFO:teuthology.orchestra.run.trial060.stdout:(77/99): python3-jaraco-text-4.0.0-2.el9.noarch 915 kB/s | 26 kB 00:00 2026-02-17T21:09:54.665 INFO:teuthology.orchestra.run.trial060.stdout:(78/99): python3-kubernetes-26.1.0-3.el9.noarch 31 MB/s | 1.0 MB 00:00 2026-02-17T21:09:54.695 INFO:teuthology.orchestra.run.trial060.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 1.5 MB/s | 46 kB 00:00 2026-02-17T21:09:54.725 INFO:teuthology.orchestra.run.trial060.stdout:(80/99): python3-more-itertools-8.12.0-2.el9.no 2.6 MB/s | 79 kB 00:00 2026-02-17T21:09:54.754 INFO:teuthology.orchestra.run.trial060.stdout:(81/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.9 MB/s | 86 kB 00:00 2026-02-17T21:09:54.783 INFO:teuthology.orchestra.run.trial060.stdout:(82/99): python3-natsort-7.1.1-5.el9.noarch.rpm 1.9 MB/s | 58 kB 00:00 2026-02-17T21:09:54.813 INFO:teuthology.orchestra.run.trial060.stdout:(83/99): python3-pecan-1.4.2-3.el9.noarch.rpm 8.8 MB/s | 272 kB 00:00 2026-02-17T21:09:54.842 INFO:teuthology.orchestra.run.trial060.stdout:(84/99): python3-portend-3.1.0-2.el9.noarch.rpm 576 kB/s | 16 kB 00:00 2026-02-17T21:09:54.872 INFO:teuthology.orchestra.run.trial060.stdout:(85/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.0 MB/s | 90 kB 00:00 2026-02-17T21:09:54.901 INFO:teuthology.orchestra.run.trial060.stdout:(86/99): python3-repoze-lru-0.7-16.el9.noarch.r 1.0 MB/s | 31 kB 00:00 2026-02-17T21:09:54.930 INFO:teuthology.orchestra.run.trial060.stdout:(87/99): python3-routes-2.5.1-5.el9.noarch.rpm 6.3 MB/s | 188 kB 00:00 2026-02-17T21:09:54.960 INFO:teuthology.orchestra.run.trial060.stdout:(88/99): python3-rsa-4.9-2.el9.noarch.rpm 1.9 MB/s | 59 kB 00:00 2026-02-17T21:09:54.990 INFO:teuthology.orchestra.run.trial060.stdout:(89/99): python3-saml-1.16.0-1.el9.noarch.rpm 4.1 MB/s | 125 kB 00:00 2026-02-17T21:09:55.019 INFO:teuthology.orchestra.run.trial060.stdout:(90/99): python3-tempora-5.0.0-2.el9.noarch.rpm 1.2 MB/s | 36 kB 00:00 2026-02-17T21:09:55.048 INFO:teuthology.orchestra.run.trial060.stdout:(91/99): python3-typing-extensions-4.15.0-1.el9 2.9 MB/s | 86 kB 00:00 2026-02-17T21:09:55.078 INFO:teuthology.orchestra.run.trial060.stdout:(92/99): python3-webob-1.8.8-2.el9.noarch.rpm 7.5 MB/s | 230 kB 00:00 2026-02-17T21:09:55.108 INFO:teuthology.orchestra.run.trial060.stdout:(93/99): python3-websocket-client-1.2.3-2.el9.n 2.9 MB/s | 90 kB 00:00 2026-02-17T21:09:55.139 INFO:teuthology.orchestra.run.trial060.stdout:(94/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 14 MB/s | 427 kB 00:00 2026-02-17T21:09:55.168 INFO:teuthology.orchestra.run.trial060.stdout:(95/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.6 MB/s | 48 kB 00:00 2026-02-17T21:09:55.197 INFO:teuthology.orchestra.run.trial060.stdout:(96/99): python3-xmltodict-0.12.0-15.el9.noarch 764 kB/s | 22 kB 00:00 2026-02-17T21:09:55.226 INFO:teuthology.orchestra.run.trial060.stdout:(97/99): python3-zc-lockfile-2.0-10.el9.noarch. 691 kB/s | 20 kB 00:00 2026-02-17T21:09:57.909 INFO:teuthology.orchestra.run.trial060.stdout:(98/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 884 kB/s | 6.1 MB 00:07 2026-02-17T21:10:04.983 INFO:teuthology.orchestra.run.trial060.stdout:(99/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 1.6 MB/s | 19 MB 00:12 2026-02-17T21:10:04.986 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:04.986 INFO:teuthology.orchestra.run.trial060.stdout:Total 4.2 MB/s | 95 MB 00:22 2026-02-17T21:10:05.385 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:05.423 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:05.423 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:05.841 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:05.841 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:06.281 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:06.283 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-17T21:10:06.314 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-17T21:10:06.319 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-17T21:10:06.341 INFO:teuthology.orchestra.run.trial060.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-17T21:10:06.347 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-17T21:10:06.380 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-17T21:10:06.390 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-17T21:10:06.395 INFO:teuthology.orchestra.run.trial060.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-17T21:10:06.397 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-17T21:10:06.402 INFO:teuthology.orchestra.run.trial060.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-17T21:10:06.425 INFO:teuthology.orchestra.run.trial060.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-17T21:10:06.438 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-17T21:10:06.441 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-17T21:10:06.447 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-17T21:10:06.449 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-17T21:10:06.460 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-17T21:10:06.467 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-17T21:10:06.491 INFO:teuthology.orchestra.run.trial060.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-17T21:10:06.495 INFO:teuthology.orchestra.run.trial060.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-17T21:10:06.502 INFO:teuthology.orchestra.run.trial060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-17T21:10:06.527 INFO:teuthology.orchestra.run.trial060.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-17T21:10:06.578 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-17T21:10:06.591 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-17T21:10:06.596 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-17T21:10:06.603 INFO:teuthology.orchestra.run.trial060.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-17T21:10:06.608 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-17T21:10:06.613 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-17T21:10:06.616 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-17T21:10:06.631 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-17T21:10:06.652 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-17T21:10:06.657 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-17T21:10:06.662 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-17T21:10:06.673 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-17T21:10:06.682 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-17T21:10:06.689 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-17T21:10:06.708 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-17T21:10:06.717 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-17T21:10:06.772 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-17T21:10:06.784 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-17T21:10:06.800 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-17T21:10:06.804 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-17T21:10:06.812 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-17T21:10:06.852 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-17T21:10:07.179 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-17T21:10:07.189 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-17T21:10:07.193 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-17T21:10:07.199 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-17T21:10:07.200 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-17T21:10:07.228 INFO:teuthology.orchestra.run.trial060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-17T21:10:07.272 INFO:teuthology.orchestra.run.trial060.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-17T21:10:07.279 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-17T21:10:07.282 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-17T21:10:07.288 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-17T21:10:07.292 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-17T21:10:07.299 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-17T21:10:07.302 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-17T21:10:07.330 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-17T21:10:07.340 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-17T21:10:07.376 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-17T21:10:07.602 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 60/99 2026-02-17T21:10:07.626 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-17T21:10:07.630 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-17T21:10:07.689 INFO:teuthology.orchestra.run.trial060.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-17T21:10:07.690 INFO:teuthology.orchestra.run.trial060.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-17T21:10:07.712 INFO:teuthology.orchestra.run.trial060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-17T21:10:08.040 INFO:teuthology.orchestra.run.trial060.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-17T21:10:08.114 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-17T21:10:08.793 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-17T21:10:08.809 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-17T21:10:08.819 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-17T21:10:08.826 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-17T21:10:08.840 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-17T21:10:08.856 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-17T21:10:08.931 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-17T21:10:08.940 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-17T21:10:08.964 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-17T21:10:08.994 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-17T21:10:09.045 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-17T21:10:09.051 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-17T21:10:09.056 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-17T21:10:09.057 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-17T21:10:09.075 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T21:10:09.075 INFO:teuthology.orchestra.run.trial060.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T21:10:09.075 INFO:teuthology.orchestra.run.trial060.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T21:10:09.075 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:09.083 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T21:10:09.109 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-17T21:10:09.109 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T21:10:09.109 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:09.144 INFO:teuthology.orchestra.run.trial060.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-17T21:10:09.208 INFO:teuthology.orchestra.run.trial060.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 84/99 2026-02-17T21:10:09.210 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-volume-2:19.2.3-679.gd855f53b.el9.noarch 85/99 2026-02-17T21:10:09.224 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-volume-2:19.2.3-679.gd855f53b.el9.noarch 85/99 2026-02-17T21:10:09.224 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:09.224 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T21:10:09.224 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:09.920 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-osd-2:19.2.3-679.gd855f53b.el9.x86_64 86/99 2026-02-17T21:10:09.945 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-osd-2:19.2.3-679.gd855f53b.el9.x86_64 86/99 2026-02-17T21:10:09.945 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:09.945 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T21:10:09.945 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T21:10:09.945 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T21:10:09.945 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:10.002 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: cephadm-2:19.2.3-679.gd855f53b.el9.noarch 87/99 2026-02-17T21:10:10.004 INFO:teuthology.orchestra.run.trial060.stdout: Installing : cephadm-2:19.2.3-679.gd855f53b.el9.noarch 87/99 2026-02-17T21:10:10.008 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-679.gd855f53b.el9. 88/99 2026-02-17T21:10:10.028 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-679.gd855f53b.el9 89/99 2026-02-17T21:10:10.029 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-679.gd855f53b.el9.noarch 90/99 2026-02-17T21:10:10.523 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-679.gd855f53b.el9.noarch 90/99 2026-02-17T21:10:10.526 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-679.gd855f53b.el9.noar 91/99 2026-02-17T21:10:10.981 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-679.gd855f53b.el9.noar 91/99 2026-02-17T21:10:10.982 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-679.gd855f5 92/99 2026-02-17T21:10:10.995 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-679.gd855f5 92/99 2026-02-17T21:10:10.995 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-679.gd855f53b.el9.noar 93/99 2026-02-17T21:10:11.052 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-679.gd855f53b.el9.noar 93/99 2026-02-17T21:10:11.101 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-679.gd855f53b.el9.n 94/99 2026-02-17T21:10:11.102 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-2:19.2.3-679.gd855f53b.el9.x86_64 95/99 2026-02-17T21:10:11.124 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-2:19.2.3-679.gd855f53b.el9.x86_64 95/99 2026-02-17T21:10:11.124 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:11.124 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T21:10:11.124 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T21:10:11.124 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T21:10:11.124 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:11.136 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mgr-rook-2:19.2.3-679.gd855f53b.el9.noarch 96/99 2026-02-17T21:10:11.352 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-679.gd855f53b.el9.noarch 96/99 2026-02-17T21:10:11.353 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mon-2:19.2.3-679.gd855f53b.el9.x86_64 97/99 2026-02-17T21:10:11.375 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mon-2:19.2.3-679.gd855f53b.el9.x86_64 97/99 2026-02-17T21:10:11.375 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:11.375 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T21:10:11.375 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T21:10:11.375 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T21:10:11.375 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:11.476 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-mds-2:19.2.3-679.gd855f53b.el9.x86_64 98/99 2026-02-17T21:10:11.497 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-mds-2:19.2.3-679.gd855f53b.el9.x86_64 98/99 2026-02-17T21:10:11.497 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:11.497 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T21:10:11.497 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T21:10:11.497 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T21:10:11.497 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:11.516 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-2:19.2.3-679.gd855f53b.el9.x86_64 99/99 2026-02-17T21:10:12.135 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-2:19.2.3-679.gd855f53b.el9.x86_64 99/99 2026-02-17T21:10:12.135 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-2:19.2.3-679.gd855f53b.el9.x86_64 1/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mds-2:19.2.3-679.gd855f53b.el9.x86_64 2/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-2:19.2.3-679.gd855f53b.el9.x86_64 3/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mon-2:19.2.3-679.gd855f53b.el9.x86_64 4/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-osd-2:19.2.3-679.gd855f53b.el9.x86_64 5/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-679.gd855f53b.el9 6/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-679.gd855f53b.el9.noarch 7/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-679.gd855f53b.el9.noar 8/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-679.gd855f5 9/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-679.gd855f53b.el9.noar 10/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-679.gd855f53b.el9.n 11/99 2026-02-17T21:10:12.136 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-mgr-rook-2:19.2.3-679.gd855f53b.el9.noarch 12/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-679.gd855f53b.el9. 13/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-volume-2:19.2.3-679.gd855f53b.el9.noarch 14/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : cephadm-2:19.2.3-679.gd855f53b.el9.noarch 15/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 16/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-17T21:10:12.137 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-17T21:10:12.138 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 39/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-17T21:10:12.139 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-17T21:10:12.140 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-17T21:10:12.141 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-17T21:10:12.142 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-17T21:10:12.143 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-17T21:10:12.198 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-17T21:10:12.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:12.198 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:12.198 INFO:teuthology.orchestra.run.trial060.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T21:10:12.198 INFO:teuthology.orchestra.run.trial060.stdout: ceph-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:12.198 INFO:teuthology.orchestra.run.trial060.stdout: ceph-grafana-dashboards-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mds-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-cephadm-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-dashboard-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-diskprediction-local-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-k8sevents-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-modules-core-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mgr-rook-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-mon-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-osd-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-prometheus-alerts-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: ceph-volume-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.199 INFO:teuthology.orchestra.run.trial060.stdout: cephadm-2:19.2.3-679.gd855f53b.el9.noarch 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T21:10:12.200 INFO:teuthology.orchestra.run.trial060.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T21:10:12.201 INFO:teuthology.orchestra.run.trial060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T21:10:12.202 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T21:10:12.203 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T21:10:12.204 INFO:teuthology.orchestra.run.trial060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-17T21:10:12.205 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: zip-3.0-35.el9.x86_64 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:12.206 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:12.253 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-base 2026-02-17T21:10:12.566 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:12.631 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-base-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:12.648 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:12.649 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:12.649 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:12.670 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install cephadm 2026-02-17T21:10:12.977 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:13.041 INFO:teuthology.orchestra.run.trial060.stdout:Package cephadm-2:19.2.3-679.gd855f53b.el9.noarch is already installed. 2026-02-17T21:10:13.058 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:13.059 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:13.059 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:13.079 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T21:10:13.386 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:13.467 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:13.468 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:13.468 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-17T21:10:13.468 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:13.468 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:13.468 INFO:teuthology.orchestra.run.trial060.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-679.gd855f53b.el9 ceph 145 k 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 145 k 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 431 k 2026-02-17T21:10:13.469 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:13.883 INFO:teuthology.orchestra.run.trial060.stdout:ceph-immutable-object-cache-19.2.3-679.gd855f53 349 kB/s | 145 kB 00:00 2026-02-17T21:10:13.884 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:13.884 INFO:teuthology.orchestra.run.trial060.stdout:Total 348 kB/s | 145 kB 00:00 2026-02-17T21:10:13.884 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:13.888 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:13.888 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:13.920 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:13.920 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:13.975 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:13.976 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-679.gd855f53b.e 1/1 2026-02-17T21:10:13.998 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-679.gd855f53b.e 1/1 2026-02-17T21:10:13.999 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:13.999 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T21:10:13.999 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:14.322 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-679.gd855f53b.e 1/1 2026-02-17T21:10:14.322 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:14.322 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:14.322 INFO:teuthology.orchestra.run.trial060.stdout: ceph-immutable-object-cache-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:14.323 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:14.323 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:14.349 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr 2026-02-17T21:10:14.655 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:14.720 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:14.737 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:14.738 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:14.738 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:14.758 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T21:10:15.067 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:03 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:15.132 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-dashboard-2:19.2.3-679.gd855f53b.el9.noarch is already installed. 2026-02-17T21:10:15.149 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:15.150 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:15.150 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:15.170 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T21:10:15.481 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:03 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:15.545 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-679.gd855f53b.el9.noarch is already installed. 2026-02-17T21:10:15.562 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:15.563 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:15.563 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:15.584 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-rook 2026-02-17T21:10:15.889 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:03 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:15.953 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-rook-2:19.2.3-679.gd855f53b.el9.noarch is already installed. 2026-02-17T21:10:15.970 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:15.971 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:15.971 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:15.992 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T21:10:16.303 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:16.367 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-mgr-cephadm-2:19.2.3-679.gd855f53b.el9.noarch is already installed. 2026-02-17T21:10:16.384 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:16.385 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:16.385 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:16.406 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-fuse 2026-02-17T21:10:16.718 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:16.799 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:16.800 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:16.800 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-17T21:10:16.800 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:16.800 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout: ceph-fuse x86_64 2:19.2.3-679.gd855f53b.el9 ceph 1.2 M 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 1.2 M 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 3.6 M 2026-02-17T21:10:16.801 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:17.570 INFO:teuthology.orchestra.run.trial060.stdout:ceph-fuse-19.2.3-679.gd855f53b.el9.x86_64.rpm 1.5 MB/s | 1.2 MB 00:00 2026-02-17T21:10:17.572 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:17.572 INFO:teuthology.orchestra.run.trial060.stdout:Total 1.5 MB/s | 1.2 MB 00:00 2026-02-17T21:10:17.572 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:17.576 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:17.576 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:17.611 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:17.612 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:17.712 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:17.729 INFO:teuthology.orchestra.run.trial060.stdout: Installing : ceph-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:17.929 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: ceph-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:17.987 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : ceph-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:17.987 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:17.988 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:17.988 INFO:teuthology.orchestra.run.trial060.stdout: ceph-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:17.988 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:17.988 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:18.016 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install ceph-volume 2026-02-17T21:10:18.330 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:06 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:18.394 INFO:teuthology.orchestra.run.trial060.stdout:Package ceph-volume-2:19.2.3-679.gd855f53b.el9.noarch is already installed. 2026-02-17T21:10:18.412 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:18.412 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:18.412 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:18.433 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install librados-devel 2026-02-17T21:10:18.743 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:06 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout: librados-devel x86_64 2:19.2.3-679.gd855f53b.el9 ceph 126 k 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:18.825 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:18.826 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:18.826 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:18.826 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:18.826 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 126 k 2026-02-17T21:10:18.826 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 456 k 2026-02-17T21:10:18.826 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:19.117 INFO:teuthology.orchestra.run.trial060.stdout:librados-devel-19.2.3-679.gd855f53b.el9.x86_64. 433 kB/s | 126 kB 00:00 2026-02-17T21:10:19.118 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:19.118 INFO:teuthology.orchestra.run.trial060.stdout:Total 432 kB/s | 126 kB 00:00 2026-02-17T21:10:19.118 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:19.121 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:19.121 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:19.152 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:19.152 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:19.207 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:19.224 INFO:teuthology.orchestra.run.trial060.stdout: Installing : librados-devel-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:19.301 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: librados-devel-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:19.359 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : librados-devel-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:19.359 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:19.359 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:19.359 INFO:teuthology.orchestra.run.trial060.stdout: librados-devel-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:19.359 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:19.359 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:19.388 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install libcephfs2 2026-02-17T21:10:19.696 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:07 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:19.760 INFO:teuthology.orchestra.run.trial060.stdout:Package libcephfs2-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:19.777 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:19.778 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:19.778 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:19.799 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install libcephfs-devel 2026-02-17T21:10:20.109 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:08 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:20.191 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-devel x86_64 2:19.2.3-679.gd855f53b.el9 ceph 34 k 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:20.192 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:20.193 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 34 k 2026-02-17T21:10:20.193 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 153 k 2026-02-17T21:10:20.193 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:20.520 INFO:teuthology.orchestra.run.trial060.stdout:libcephfs-devel-19.2.3-679.gd855f53b.el9.x86_64 102 kB/s | 34 kB 00:00 2026-02-17T21:10:20.521 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:20.521 INFO:teuthology.orchestra.run.trial060.stdout:Total 102 kB/s | 34 kB 00:00 2026-02-17T21:10:20.521 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:20.523 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:20.524 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:20.526 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:20.526 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:20.549 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:20.565 INFO:teuthology.orchestra.run.trial060.stdout: Installing : libcephfs-devel-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:20.619 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:20.675 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : libcephfs-devel-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:20.675 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:20.675 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:20.675 INFO:teuthology.orchestra.run.trial060.stdout: libcephfs-devel-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:20.675 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:20.675 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:20.701 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install librados2 2026-02-17T21:10:21.010 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:21.074 INFO:teuthology.orchestra.run.trial060.stdout:Package librados2-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:21.091 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:21.092 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:21.092 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:21.113 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install librbd1 2026-02-17T21:10:21.420 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:21.483 INFO:teuthology.orchestra.run.trial060.stdout:Package librbd1-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:21.501 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:21.502 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:21.502 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:21.524 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-rados 2026-02-17T21:10:21.831 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:21.894 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-rados-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:21.911 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:21.912 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:21.912 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:21.933 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-rgw 2026-02-17T21:10:22.238 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:10 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:22.301 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-rgw-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:22.319 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:22.319 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:22.320 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:22.341 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-cephfs 2026-02-17T21:10:22.654 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:10 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:22.719 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-cephfs-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:22.737 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:22.737 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:22.738 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:22.759 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install python3-rbd 2026-02-17T21:10:23.068 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:11 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:23.132 INFO:teuthology.orchestra.run.trial060.stdout:Package python3-rbd-2:19.2.3-679.gd855f53b.el9.x86_64 is already installed. 2026-02-17T21:10:23.150 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:23.150 INFO:teuthology.orchestra.run.trial060.stdout:Nothing to do. 2026-02-17T21:10:23.151 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:23.173 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install rbd-fuse 2026-02-17T21:10:23.483 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:11 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout: rbd-fuse x86_64 2:19.2.3-679.gd855f53b.el9 ceph 85 k 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:23.566 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:23.567 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:23.567 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:23.567 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:23.567 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 85 k 2026-02-17T21:10:23.567 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 227 k 2026-02-17T21:10:23.567 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:23.802 INFO:teuthology.orchestra.run.trial060.stdout:rbd-fuse-19.2.3-679.gd855f53b.el9.x86_64.rpm 361 kB/s | 85 kB 00:00 2026-02-17T21:10:23.803 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:23.803 INFO:teuthology.orchestra.run.trial060.stdout:Total 359 kB/s | 85 kB 00:00 2026-02-17T21:10:23.803 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:23.807 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:23.807 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:23.838 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:23.838 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:23.889 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:23.906 INFO:teuthology.orchestra.run.trial060.stdout: Installing : rbd-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:23.982 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: rbd-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:24.039 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : rbd-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:24.039 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:24.039 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:24.039 INFO:teuthology.orchestra.run.trial060.stdout: rbd-fuse-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:24.039 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:24.040 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:24.067 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install rbd-mirror 2026-02-17T21:10:24.381 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:12 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:24.463 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:24.463 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:24.463 INFO:teuthology.orchestra.run.trial060.stdout: Package Arch Version Repo Size 2026-02-17T21:10:24.463 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:24.463 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:24.463 INFO:teuthology.orchestra.run.trial060.stdout: rbd-mirror x86_64 2:19.2.3-679.gd855f53b.el9 ceph 3.1 M 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 3.1 M 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 13 M 2026-02-17T21:10:24.464 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:25.422 INFO:teuthology.orchestra.run.trial060.stdout:rbd-mirror-19.2.3-679.gd855f53b.el9.x86_64.rpm 3.3 MB/s | 3.1 MB 00:00 2026-02-17T21:10:25.423 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:25.424 INFO:teuthology.orchestra.run.trial060.stdout:Total 3.2 MB/s | 3.1 MB 00:00 2026-02-17T21:10:25.424 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:25.428 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:25.428 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:25.468 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:25.468 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:25.660 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:25.662 INFO:teuthology.orchestra.run.trial060.stdout: Installing : rbd-mirror-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:25.685 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: rbd-mirror-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:25.685 INFO:teuthology.orchestra.run.trial060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T21:10:25.685 INFO:teuthology.orchestra.run.trial060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T21:10:25.685 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T21:10:25.685 INFO:teuthology.orchestra.run.trial060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T21:10:25.685 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:25.981 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : rbd-mirror-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:25.981 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:25.981 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:25.981 INFO:teuthology.orchestra.run.trial060.stdout: rbd-mirror-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:25.981 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:25.981 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:26.007 DEBUG:teuthology.orchestra.run.trial060:> sudo yum -y install rbd-nbd 2026-02-17T21:10:26.316 INFO:teuthology.orchestra.run.trial060.stdout:Last metadata expiration check: 0:01:14 ago on Tue 17 Feb 2026 09:09:12 PM UTC. 2026-02-17T21:10:26.400 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-17T21:10:26.400 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout: rbd-nbd x86_64 2:19.2.3-679.gd855f53b.el9 ceph 170 k 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:Install 1 Package 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 170 k 2026-02-17T21:10:26.401 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 490 k 2026-02-17T21:10:26.402 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-17T21:10:26.592 INFO:teuthology.orchestra.run.trial060.stdout:rbd-nbd-19.2.3-679.gd855f53b.el9.x86_64.rpm 888 kB/s | 170 kB 00:00 2026-02-17T21:10:26.593 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-17T21:10:26.593 INFO:teuthology.orchestra.run.trial060.stdout:Total 884 kB/s | 170 kB 00:00 2026-02-17T21:10:26.593 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-17T21:10:26.597 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-17T21:10:26.597 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-17T21:10:26.629 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-17T21:10:26.629 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-17T21:10:26.685 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-17T21:10:26.702 INFO:teuthology.orchestra.run.trial060.stdout: Installing : rbd-nbd-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:26.779 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: rbd-nbd-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:26.837 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : rbd-nbd-2:19.2.3-679.gd855f53b.el9.x86_64 1/1 2026-02-17T21:10:26.837 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:26.837 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-17T21:10:26.837 INFO:teuthology.orchestra.run.trial060.stdout: rbd-nbd-2:19.2.3-679.gd855f53b.el9.x86_64 2026-02-17T21:10:26.837 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:26.837 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-17T21:10:26.868 DEBUG:teuthology.parallel:result is None 2026-02-17T21:10:26.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:10:27.034 DEBUG:teuthology.orchestra.run.trial060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T21:10:27.054 INFO:teuthology.orchestra.run.trial060.stdout:19.2.3-679.gd855f53b.el9 2026-02-17T21:10:27.055 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-679.gd855f53b.el9 2026-02-17T21:10:27.055 INFO:teuthology.task.install:The correct ceph version 19.2.3-679.gd855f53b is installed. 2026-02-17T21:10:27.057 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T21:10:27.058 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:27.058 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T21:10:27.119 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T21:10:27.120 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:27.120 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T21:10:27.185 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T21:10:27.249 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T21:10:27.249 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:27.249 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T21:10:27.314 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T21:10:27.379 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T21:10:27.379 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:27.380 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T21:10:27.444 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T21:10:27.509 INFO:teuthology.run_tasks:Running task ceph... 2026-02-17T21:10:27.623 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-17T21:10:27.623 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 777 /var/log/ceph 2026-02-17T21:10:27.646 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-17T21:10:27.646 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T21:10:27.710 INFO:tasks.ceph:Creating extra log directories... 2026-02-17T21:10:27.710 DEBUG:teuthology.orchestra.run.trial060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T21:10:27.787 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-17T21:10:27.788 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'overall HEALTH_', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-17T21:10:27.788 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889', 'branch': 'wip-yuri6-testing-2026-02-17-1732-squid', 'description': 'rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_latest}}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '53889', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-17-1732-squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'overall HEALTH_', '\\(POOL_APP_NOT_ENABLED\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-17-1732-squid', 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2987, 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2026-02-17-1732-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1', 'targets': {'trial060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAEYQ3k/YawAFqDLdhxcMEz2lbT1ozEE2FaEYVxtNa50CLoN+S4GTQSB0kjdFrZDBuKz5KXHBQybE1+k4Y7jAE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'overall HEALTH_', '\\(POOL_APP_NOT_ENABLED\\)'], 'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['cephtool/test_kvstore_tool.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': '5f66ecfb34c0370410e78b3ee641753d19da653b', 'timestamp': '2026-02-17_20:43:43', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682'} 2026-02-17T21:10:27.788 DEBUG:teuthology.orchestra.run.trial060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T21:10:27.833 DEBUG:teuthology.orchestra.run.trial060:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T21:10:27.899 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:27.899 DEBUG:teuthology.orchestra.run.trial060:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T21:10:27.957 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T21:10:27.957 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_1 2026-02-17T21:10:28.013 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T21:10:28.013 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1311 Links: 1 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-17 21:10:25.828903815 +0000 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-17 21:09:20.332592439 +0000 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-17 21:09:20.332592439 +0000 2026-02-17T21:10:28.014 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-17 21:09:20.332592439 +0000 2026-02-17T21:10:28.015 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T21:10:28.096 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-17T21:10:28.097 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-17T21:10:28.097 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.000231471 s, 2.2 MB/s 2026-02-17T21:10:28.098 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T21:10:28.173 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_2 2026-02-17T21:10:28.231 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T21:10:28.231 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T21:10:28.231 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-17T21:10:28.232 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T21:10:28.232 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T21:10:28.232 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-17 21:10:25.829903787 +0000 2026-02-17T21:10:28.232 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.232 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.232 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.232 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T21:10:28.294 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-17T21:10:28.295 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-17T21:10:28.295 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.000221361 s, 2.3 MB/s 2026-02-17T21:10:28.296 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T21:10:28.356 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_3 2026-02-17T21:10:28.412 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-17 21:10:25.829903787 +0000 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.413 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.414 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T21:10:28.475 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-17T21:10:28.475 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-17T21:10:28.476 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.000146222 s, 3.5 MB/s 2026-02-17T21:10:28.477 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T21:10:28.533 DEBUG:teuthology.orchestra.run.trial060:> stat /dev/vg_nvme/lv_4 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout:Device: 6h/6d Inode: 1291 Links: 1 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout:Access: 2026-02-17 21:10:25.829903787 +0000 2026-02-17T21:10:28.590 INFO:teuthology.orchestra.run.trial060.stdout:Modify: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.591 INFO:teuthology.orchestra.run.trial060.stdout:Change: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.591 INFO:teuthology.orchestra.run.trial060.stdout: Birth: 2026-02-17 21:09:20.331592464 +0000 2026-02-17T21:10:28.591 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T21:10:28.652 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records in 2026-02-17T21:10:28.652 INFO:teuthology.orchestra.run.trial060.stderr:1+0 records out 2026-02-17T21:10:28.653 INFO:teuthology.orchestra.run.trial060.stderr:512 bytes copied, 0.00013965 s, 3.7 MB/s 2026-02-17T21:10:28.654 DEBUG:teuthology.orchestra.run.trial060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T21:10:28.710 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T21:10:28.711 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-17T21:10:28.711 INFO:tasks.ceph:Generating config... 2026-02-17T21:10:28.713 INFO:tasks.ceph:[global] mon election default strategy = 3 2026-02-17T21:10:28.713 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-17T21:10:28.713 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-17T21:10:28.714 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-17T21:10:28.714 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-17T21:10:28.714 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-17T21:10:28.714 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-17T21:10:28.714 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-17T21:10:28.715 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-17T21:10:28.715 INFO:tasks.ceph:Setting up mon.a... 2026-02-17T21:10:28.715 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-17T21:10:28.785 INFO:teuthology.orchestra.run.trial060.stdout:creating /etc/ceph/ceph.keyring 2026-02-17T21:10:28.787 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-17T21:10:28.863 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T21:10:28.928 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.60')] 2026-02-17T21:10:28.928 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.60', 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-17T21:10:28.930 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:28.930 DEBUG:teuthology.orchestra.run.trial060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T21:10:28.984 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.60 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:29.056 INFO:teuthology.orchestra.run.trial060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:monmaptool: generated fsid 693a7425-f499-41f0-a5e5-0a2aa082bd73 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:setting min_mon_release = quincy 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:epoch 0 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:fsid 693a7425-f499-41f0-a5e5-0a2aa082bd73 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:last_changed 2026-02-17T21:10:29.055887+0000 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:created 2026-02-17T21:10:29.055887+0000 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:min_mon_release 17 (quincy) 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:election_strategy: 3 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:0: [v2:10.20.193.60:3300/0,v1:10.20.193.60:6789/0] mon.a 2026-02-17T21:10:29.057 INFO:teuthology.orchestra.run.trial060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-17T21:10:29.059 DEBUG:teuthology.orchestra.run.trial060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T21:10:29.115 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 693a7425-f499-41f0-a5e5-0a2aa082bd73... 2026-02-17T21:10:29.116 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T21:10:29.193 INFO:teuthology.orchestra.run.trial060.stdout:[global] 2026-02-17T21:10:29.193 INFO:teuthology.orchestra.run.trial060.stdout: chdir = "" 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: auth supported = cephx 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: filestore xattr use omap = true 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: mon clock drift allowed = 1.000 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: osd crush chooseleaf type = 0 2026-02-17T21:10:29.194 INFO:teuthology.orchestra.run.trial060.stdout: auth debug = true 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: ms die on old message = true 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: ms die on bug = true 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: mon pg warn max object skew = 0 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: osd pool default size = 2 2026-02-17T21:10:29.195 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon osd allow primary affinity = true 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon osd allow pg remap = true 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on legacy crush tunables = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on crush straw calc version zero = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on no sortbitwise = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on osd down out interval zero = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon warn on too few osds = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: mon_allow_pool_size_one = true 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.196 INFO:teuthology.orchestra.run.trial060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: osd default data pool replay window = 5 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: mon allow pool delete = true 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: mon cluster log file level = debug 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: debug asserts on shutdown = true 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: mon health detail to clog = false 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: mon host = 10.20.193.60 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: mon election default strategy = 3 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: fsid = 693a7425-f499-41f0-a5e5-0a2aa082bd73 2026-02-17T21:10:29.197 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout:[osd] 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd journal size = 100 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd scrub load threshold = 5.0 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd scrub max interval = 600 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd mclock profile = high_recovery_ops 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd recover clone overlap = true 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd recovery max chunk = 1048576 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd debug shutdown = true 2026-02-17T21:10:29.198 INFO:teuthology.orchestra.run.trial060.stdout: osd debug op order = true 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: osd debug verify stray on activate = true 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: osd debug trim objects = true 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: osd open classes on start = true 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: osd debug pg log writeout = true 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: osd deep scrub update digest min age = 30 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: osd map max advance = 10 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: journal zero on create = true 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: filestore ondisk finisher threads = 3 2026-02-17T21:10:29.199 INFO:teuthology.orchestra.run.trial060.stdout: filestore apply finisher threads = 3 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: bdev debug aio = true 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: osd debug misdirected ops = true 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: bluestore zero block detection = True 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: debug ms = 1 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: debug osd = 20 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: osd debug verify cached snaps = True 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: osd debug verify missing on start = True 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: osd mclock override recovery settings = True 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: osd op queue = debug_random 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: osd op queue cut off = debug_random 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.200 INFO:teuthology.orchestra.run.trial060.stdout:[mgr] 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: debug ms = 1 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: debug mgr = 20 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: debug mon = 20 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: debug auth = 20 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min pgs per osd = 4 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min bytes per osd = 10 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: mgr/telemetry/nag = false 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout:[mon] 2026-02-17T21:10:29.201 INFO:teuthology.orchestra.run.trial060.stdout: debug ms = 1 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: debug mon = 20 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: debug paxos = 20 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: debug auth = 20 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: mon data avail warn = 5 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: mon mgr mkfs grace = 240 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min pgs per osd = 4 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: mon osd reporter subtree level = osd 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: mon osd prime pg temp = true 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: mon reweight min bytes per osd = 10 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.202 INFO:teuthology.orchestra.run.trial060.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: # 1m isn't quite enough 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: mon_down_mkfs_grace = 2m 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.203 INFO:teuthology.orchestra.run.trial060.stdout: mon_warn_on_filestore_osds = false 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: mon scrub interval = 300 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout:[client] 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: rgw cache enabled = true 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: rgw enable ops log = true 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: rgw enable usage log = true 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T21:10:29.204 INFO:teuthology.orchestra.run.trial060.stdout:[mon.a] 2026-02-17T21:10:29.206 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-17T21:10:29.206 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-17T21:10:29.286 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-17T21:10:29.287 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:29.287 DEBUG:teuthology.orchestra.run.trial060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-17T21:10:29.345 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:29.345 DEBUG:teuthology.orchestra.run.trial060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-17T21:10:29.401 INFO:tasks.ceph:Sending monmap to node ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:10:29.401 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:29.401 DEBUG:teuthology.orchestra.run.trial060:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T21:10:29.401 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T21:10:29.473 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:29.473 DEBUG:teuthology.orchestra.run.trial060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:29.529 INFO:tasks.ceph:Setting up mon nodes... 2026-02-17T21:10:29.529 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-17T21:10:29.529 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T21:10:29.613 INFO:teuthology.orchestra.run.trial060.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T21:10:29.615 INFO:tasks.ceph:Setting up mds nodes... 2026-02-17T21:10:29.615 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-17T21:10:29.616 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-17T21:10:29.648 INFO:teuthology.orchestra.run.trial060.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-17T21:10:29.659 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-17T21:10:29.659 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-17T21:10:29.660 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-17T21:10:29.726 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T21:10:29.726 INFO:tasks.ceph:role: osd.0 2026-02-17T21:10:29.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:10:29.727 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T21:10:29.791 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T21:10:29.791 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T21:10:29.791 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T21:10:29.791 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T21:10:29.791 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T21:10:29.792 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-17T21:10:29.792 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T21:10:29.792 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T21:10:29.792 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T21:10:29.792 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T21:10:30.368 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-17T21:10:30.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-17T21:10:30.371 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-17T21:10:30.440 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-17T21:10:30.508 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-17T21:10:30.572 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T21:10:30.572 INFO:tasks.ceph:role: osd.1 2026-02-17T21:10:30.572 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:10:30.573 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T21:10:30.636 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T21:10:30.637 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T21:10:31.200 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-17T21:10:31.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-17T21:10:31.204 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-17T21:10:31.274 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-17T21:10:31.340 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-17T21:10:31.404 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T21:10:31.404 INFO:tasks.ceph:role: osd.2 2026-02-17T21:10:31.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:10:31.405 DEBUG:teuthology.orchestra.run.trial060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T21:10:31.469 INFO:teuthology.orchestra.run.trial060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T21:10:31.469 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T21:10:31.469 INFO:teuthology.orchestra.run.trial060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T21:10:31.469 INFO:teuthology.orchestra.run.trial060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T21:10:31.470 INFO:teuthology.orchestra.run.trial060.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T21:10:31.470 INFO:teuthology.orchestra.run.trial060.stdout: = sunit=0 swidth=0 blks 2026-02-17T21:10:31.470 INFO:teuthology.orchestra.run.trial060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T21:10:31.470 INFO:teuthology.orchestra.run.trial060.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T21:10:31.470 INFO:teuthology.orchestra.run.trial060.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T21:10:31.470 INFO:teuthology.orchestra.run.trial060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T21:10:32.028 INFO:teuthology.orchestra.run.trial060.stdout:Discarding blocks...Done. 2026-02-17T21:10:32.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial060.front.sepia.ceph.com -o noatime 2026-02-17T21:10:32.031 DEBUG:teuthology.orchestra.run.trial060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-17T21:10:32.103 DEBUG:teuthology.orchestra.run.trial060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-17T21:10:32.169 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:32.247 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:32.245+0000 7f4c2192e780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-17T21:10:32.247 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:32.245+0000 7f4c2192e780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-17T21:10:32.247 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:32.245+0000 7f4c2192e780 -1 bdev(0x55d9e917b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-17T21:10:32.247 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:32.245+0000 7f4c2192e780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-17T21:10:34.360 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-17T21:10:34.426 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:34.504 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:34.503+0000 7f30a060e780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-17T21:10:34.505 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:34.503+0000 7f30a060e780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-17T21:10:34.505 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:34.503+0000 7f30a060e780 -1 bdev(0x556af9af1800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-17T21:10:34.505 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:34.503+0000 7f30a060e780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-17T21:10:36.878 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-17T21:10:36.944 DEBUG:teuthology.orchestra.run.trial060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:37.021 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:37.018+0000 7fc941218780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-17T21:10:37.021 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:37.019+0000 7fc941218780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-17T21:10:37.021 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:37.019+0000 7fc941218780 -1 bdev(0x55c0ddb1d800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-17T21:10:37.021 INFO:teuthology.orchestra.run.trial060.stderr:2026-02-17T21:10:37.019+0000 7fc941218780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-17T21:10:39.378 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-17T21:10:39.443 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-17T21:10:39.443 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:39.444 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-17T21:10:39.508 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:39.508 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-17T21:10:39.571 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:39.571 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-17T21:10:39.634 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:39.634 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-17T21:10:39.697 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:39.697 DEBUG:teuthology.orchestra.run.trial060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-17T21:10:39.754 INFO:tasks.ceph:Adding keys to all mons... 2026-02-17T21:10:39.754 DEBUG:teuthology.orchestra.run.trial060:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T21:10:39.815 INFO:teuthology.orchestra.run.trial060.stdout:[mgr.x] 2026-02-17T21:10:39.815 INFO:teuthology.orchestra.run.trial060.stdout: key = AQBF2ZRpAGB/JBAAYMYxRF8nHuOz1mfWZ9+yow== 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout:[osd.0] 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout: key = AQBI2ZRpQQ+tDhAAXS5DCHNechPQUh3f9i9oNQ== 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout:[osd.1] 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout: key = AQBK2ZRpaTwTHhAAvBQpU1wLoO2im/cXVfr7vg== 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout:[osd.2] 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout: key = AQBN2ZRpH1MxARAAZ7sDhu49XfuAXwr1D/cwyQ== 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout:[client.0] 2026-02-17T21:10:39.816 INFO:teuthology.orchestra.run.trial060.stdout: key = AQBF2ZRp4GCbJhAATusWADTn42Nqpt8hJXzjNw== 2026-02-17T21:10:39.817 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T21:10:39.897 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T21:10:39.976 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T21:10:40.056 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T21:10:40.133 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T21:10:40.214 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-17T21:10:40.215 DEBUG:teuthology.orchestra.run.trial060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-17T21:10:40.280 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T21:10:40.361 DEBUG:teuthology.orchestra.run.trial060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-17T21:10:40.425 DEBUG:teuthology.orchestra.run.trial060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T21:10:40.482 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-17T21:10:40.482 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T21:10:40.482 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T21:10:40.524 INFO:tasks.ceph.mon.a:Started 2026-02-17T21:10:40.524 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph config set mgr mgr_pool false --force 2026-02-17T21:10:40.633 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-17T21:10:40.633 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-17T21:10:40.633 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-17T21:10:40.636 INFO:tasks.ceph.mgr.x:Started 2026-02-17T21:10:40.636 INFO:tasks.ceph:Setting crush tunables to default 2026-02-17T21:10:40.636 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd crush tunables default 2026-02-17T21:10:40.734 INFO:teuthology.orchestra.run.trial060.stderr:adjusted tunables profile to default 2026-02-17T21:10:40.742 INFO:tasks.ceph:check_enable_crimson: False 2026-02-17T21:10:40.742 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-17T21:10:40.743 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:40.743 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-17T21:10:40.766 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:40.766 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-17T21:10:40.832 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:10:40.832 DEBUG:teuthology.orchestra.run.trial060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-17T21:10:40.896 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new 1314774c-e40f-425a-9715-3d5104938d9e 0 2026-02-17T21:10:41.038 INFO:teuthology.orchestra.run.trial060.stdout:0 2026-02-17T21:10:41.048 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new 416b341c-8e2f-4371-a402-e379d3799a48 1 2026-02-17T21:10:41.048 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.046+0000 7fbc3d3da140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-17T21:10:41.151 INFO:teuthology.orchestra.run.trial060.stdout:1 2026-02-17T21:10:41.160 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd new d0157467-2bcb-4a82-aa65-a9f16d18a519 2 2026-02-17T21:10:41.172 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.171+0000 7fbc3d3da140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-17T21:10:41.200 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.199+0000 7fbc3d3da140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-17T21:10:41.234 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.233+0000 7fbc3d3da140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-17T21:10:41.258 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.256+0000 7fbc3d3da140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-17T21:10:41.265 INFO:teuthology.orchestra.run.trial060.stdout:2 2026-02-17T21:10:41.274 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-17T21:10:41.275 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-17T21:10:41.286 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.284+0000 7fbc3d3da140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-17T21:10:41.287 INFO:tasks.ceph.osd.0:Started 2026-02-17T21:10:41.288 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-17T21:10:41.288 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-17T21:10:41.290 INFO:tasks.ceph.osd.1:Started 2026-02-17T21:10:41.290 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-17T21:10:41.290 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-17T21:10:41.292 INFO:tasks.ceph.osd.2:Started 2026-02-17T21:10:41.293 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T21:10:41.346 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.344+0000 7fbc3d3da140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-17T21:10:41.392 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:41.392 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":5,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:10:41.264747+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:10:41.398 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-17T21:10:41.398 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-17T21:10:41.410 INFO:tasks.ceph.mgr.x.trial060.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-17T21:10:41.410 INFO:tasks.ceph.mgr.x.trial060.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-17T21:10:41.411 INFO:tasks.ceph.mgr.x.trial060.stderr: from numpy import show_config as show_numpy_config 2026-02-17T21:10:41.414 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.412+0000 7fbc3d3da140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-17T21:10:41.440 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.438+0000 7fbc3d3da140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-17T21:10:41.496 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.495+0000 7fbc3d3da140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-17T21:10:41.541 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.539+0000 7fbc3d3da140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-17T21:10:41.567 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.565+0000 7fbc3d3da140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-17T21:10:41.591 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.590+0000 7fbc3d3da140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-17T21:10:41.618 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.616+0000 7fbc3d3da140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-17T21:10:41.641 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.639+0000 7fbc3d3da140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-17T21:10:41.700 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T21:10:41.792 INFO:teuthology.misc.health.trial060.stdout: 2026-02-17T21:10:41.792 INFO:teuthology.misc.health.trial060.stdout:{"epoch":5,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:10:41.264747+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:10:41.797 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-17T21:10:41.842 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.841+0000 7fbc3d3da140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-17T21:10:41.898 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.896+0000 7fbc3d3da140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-17T21:10:41.920 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.919+0000 7fbc3d3da140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-17T21:10:41.944 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.942+0000 7fbc3d3da140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-17T21:10:41.998 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:41.996+0000 7fbc3d3da140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-17T21:10:42.019 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:42.017+0000 7fbc3d3da140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-17T21:10:42.072 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:42.070+0000 7fbc3d3da140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-17T21:10:42.466 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:42.464+0000 7fbc3d3da140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-17T21:10:42.488 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:42.487+0000 7fbc3d3da140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-17T21:10:42.726 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-17T21:10:42.724+0000 7f262a212780 -1 Falling back to public interface 2026-02-17T21:10:42.829 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:42.827+0000 7fbc3d3da140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-17T21:10:42.986 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-17T21:10:42.984+0000 7f34a6f64780 -1 Falling back to public interface 2026-02-17T21:10:43.217 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-17T21:10:43.215+0000 7fc47b80d780 -1 Falling back to public interface 2026-02-17T21:10:43.834 INFO:tasks.ceph.mgr.x.trial060.stderr:2026-02-17T21:10:43.832+0000 7fbb5a126640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-17T21:10:45.441 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-17T21:10:45.439+0000 7f262a212780 -1 osd.2 0 log_to_monitors true 2026-02-17T21:10:45.701 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-17T21:10:45.700+0000 7f34a6f64780 -1 osd.0 0 log_to_monitors true 2026-02-17T21:10:46.163 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-17T21:10:46.162+0000 7fc47b80d780 -1 osd.1 0 log_to_monitors true 2026-02-17T21:10:46.852 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-17T21:10:46.851+0000 7f34a2ef3640 -1 osd.0 0 waiting for initial osdmap 2026-02-17T21:10:46.854 INFO:tasks.ceph.osd.0.trial060.stderr:2026-02-17T21:10:46.852+0000 7f349ed1d640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T21:10:47.027 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-17T21:10:47.025+0000 7f2625b49640 -1 osd.2 0 waiting for initial osdmap 2026-02-17T21:10:47.036 INFO:tasks.ceph.osd.2.trial060.stderr:2026-02-17T21:10:47.035+0000 7f261d96b640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T21:10:48.036 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-17T21:10:48.034+0000 7fc477080640 -1 osd.1 0 waiting for initial osdmap 2026-02-17T21:10:48.045 INFO:tasks.ceph.osd.1.trial060.stderr:2026-02-17T21:10:48.044+0000 7fc46eea2640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-17T21:10:48.100 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T21:10:48.248 INFO:teuthology.misc.health.trial060.stdout: 2026-02-17T21:10:48.249 INFO:teuthology.misc.health.trial060.stdout:{"epoch":8,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:10:47.836494+0000","last_up_change":"2026-02-17T21:10:47.836494+0000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6811","nonce":1320383713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6813","nonce":1320383713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6817","nonce":1320383713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6814","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6815","nonce":1320383713}]},"public_addr":"10.20.193.60:6811/1320383713","cluster_addr":"10.20.193.60:6813/1320383713","heartbeat_back_addr":"10.20.193.60:6817/1320383713","heartbeat_front_addr":"10.20.193.60:6815/1320383713","state":["exists","up"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6800","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6801","nonce":633340544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6803","nonce":633340544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6807","nonce":633340544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6805","nonce":633340544}]},"public_addr":"10.20.193.60:6801/633340544","cluster_addr":"10.20.193.60:6803/633340544","heartbeat_back_addr":"10.20.193.60:6807/633340544","heartbeat_front_addr":"10.20.193.60:6805/633340544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:10:48.255 DEBUG:teuthology.misc:2 of 3 OSDs are up 2026-02-17T21:10:54.557 DEBUG:teuthology.orchestra.run.trial060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T21:10:54.708 INFO:teuthology.misc.health.trial060.stdout: 2026-02-17T21:10:54.708 INFO:teuthology.misc.health.trial060.stdout:{"epoch":9,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:10:49.035977+0000","last_up_change":"2026-02-17T21:10:49.035977+0000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6811","nonce":1320383713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6813","nonce":1320383713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6817","nonce":1320383713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6814","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6815","nonce":1320383713}]},"public_addr":"10.20.193.60:6811/1320383713","cluster_addr":"10.20.193.60:6813/1320383713","heartbeat_back_addr":"10.20.193.60:6817/1320383713","heartbeat_front_addr":"10.20.193.60:6815/1320383713","state":["exists","up"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6819","nonce":2878742521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6820","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6821","nonce":2878742521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6824","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6825","nonce":2878742521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6823","nonce":2878742521}]},"public_addr":"10.20.193.60:6819/2878742521","cluster_addr":"10.20.193.60:6821/2878742521","heartbeat_back_addr":"10.20.193.60:6825/2878742521","heartbeat_front_addr":"10.20.193.60:6823/2878742521","state":["exists","up"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6800","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6801","nonce":633340544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6803","nonce":633340544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6807","nonce":633340544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6805","nonce":633340544}]},"public_addr":"10.20.193.60:6801/633340544","cluster_addr":"10.20.193.60:6803/633340544","heartbeat_back_addr":"10.20.193.60:6807/633340544","heartbeat_front_addr":"10.20.193.60:6805/633340544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:10:54.715 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-17T21:10:54.716 INFO:tasks.ceph:Creating RBD pool 2026-02-17T21:10:54.716 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-17T21:10:55.042 INFO:teuthology.orchestra.run.trial060.stderr:pool 'rbd' created 2026-02-17T21:10:55.053 DEBUG:teuthology.orchestra.run.trial060:> rbd --cluster ceph pool init rbd 2026-02-17T21:10:58.055 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-17T21:10:58.056 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-17T21:10:58.056 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-17T21:10:58.056 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-17T21:10:58.057 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-17T21:10:58.234 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:58.242 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6808","nonce":2689727324},{"type":"v1","addr":"10.20.193.60:6809","nonce":2689727324}]},"active_addr":"10.20.193.60:6809/2689727324","active_change":"2026-02-17T21:10:42.829432+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.60:0","nonce":3811262794}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.60:0","nonce":2345173125}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.60:0","nonce":2540637033}]}]} 2026-02-17T21:10:58.244 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-17T21:10:58.244 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-17T21:10:58.244 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T21:10:58.403 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:58.403 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":13,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:10:58.046064+0000","last_up_change":"2026-02-17T21:10:49.035977+0000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-17T21:10:54.874321+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6811","nonce":1320383713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6813","nonce":1320383713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6817","nonce":1320383713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6814","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6815","nonce":1320383713}]},"public_addr":"10.20.193.60:6811/1320383713","cluster_addr":"10.20.193.60:6813/1320383713","heartbeat_back_addr":"10.20.193.60:6817/1320383713","heartbeat_front_addr":"10.20.193.60:6815/1320383713","state":["exists","up"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6819","nonce":2878742521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6820","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6821","nonce":2878742521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6824","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6825","nonce":2878742521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6823","nonce":2878742521}]},"public_addr":"10.20.193.60:6819/2878742521","cluster_addr":"10.20.193.60:6821/2878742521","heartbeat_back_addr":"10.20.193.60:6825/2878742521","heartbeat_front_addr":"10.20.193.60:6823/2878742521","state":["exists","up"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6800","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6801","nonce":633340544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6803","nonce":633340544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6807","nonce":633340544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6805","nonce":633340544}]},"public_addr":"10.20.193.60:6801/633340544","cluster_addr":"10.20.193.60:6803/633340544","heartbeat_back_addr":"10.20.193.60:6807/633340544","heartbeat_front_addr":"10.20.193.60:6805/633340544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:10:58.412 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-17T21:10:58.413 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T21:10:58.570 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:10:58.570 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":13,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:10:58.046064+0000","last_up_change":"2026-02-17T21:10:49.035977+0000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-17T21:10:54.874321+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6811","nonce":1320383713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6813","nonce":1320383713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6817","nonce":1320383713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6814","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6815","nonce":1320383713}]},"public_addr":"10.20.193.60:6811/1320383713","cluster_addr":"10.20.193.60:6813/1320383713","heartbeat_back_addr":"10.20.193.60:6817/1320383713","heartbeat_front_addr":"10.20.193.60:6815/1320383713","state":["exists","up"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6819","nonce":2878742521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6820","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6821","nonce":2878742521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6824","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6825","nonce":2878742521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6823","nonce":2878742521}]},"public_addr":"10.20.193.60:6819/2878742521","cluster_addr":"10.20.193.60:6821/2878742521","heartbeat_back_addr":"10.20.193.60:6825/2878742521","heartbeat_front_addr":"10.20.193.60:6823/2878742521","state":["exists","up"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6800","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6801","nonce":633340544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6803","nonce":633340544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6807","nonce":633340544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6805","nonce":633340544}]},"public_addr":"10.20.193.60:6801/633340544","cluster_addr":"10.20.193.60:6803/633340544","heartbeat_back_addr":"10.20.193.60:6807/633340544","heartbeat_front_addr":"10.20.193.60:6805/633340544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:10:58.579 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-17T21:10:58.580 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-17T21:10:58.580 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-17T21:10:58.666 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-17T21:10:58.666 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T21:10:58.667 INFO:teuthology.orchestra.run.trial060.stdout:34359738373 2026-02-17T21:10:58.668 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T21:10:58.669 INFO:teuthology.orchestra.run.trial060.stdout:34359738373 2026-02-17T21:10:58.669 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T21:10:58.866 INFO:teuthology.orchestra.run.trial060.stdout:34359738371 2026-02-17T21:10:58.868 INFO:teuthology.orchestra.run.trial060.stdout:38654705668 2026-02-17T21:10:58.871 INFO:teuthology.orchestra.run.trial060.stdout:34359738371 2026-02-17T21:10:58.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.0 2026-02-17T21:10:58.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.1 2026-02-17T21:10:58.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.2 2026-02-17T21:10:59.876 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T21:10:59.878 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T21:10:59.881 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T21:11:00.039 INFO:teuthology.orchestra.run.trial060.stdout:38654705669 2026-02-17T21:11:00.044 INFO:teuthology.orchestra.run.trial060.stdout:34359738373 2026-02-17T21:11:00.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2026-02-17T21:11:00.050 DEBUG:teuthology.parallel:result is None 2026-02-17T21:11:00.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.0 2026-02-17T21:11:00.054 DEBUG:teuthology.parallel:result is None 2026-02-17T21:11:00.077 INFO:teuthology.orchestra.run.trial060.stdout:34359738373 2026-02-17T21:11:00.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2026-02-17T21:11:00.087 DEBUG:teuthology.parallel:result is None 2026-02-17T21:11:00.088 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T21:11:00.088 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:11:00.282 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:00.282 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:11:00.290 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-17T21:10:58.833472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81240,"kb_used_data":336,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491560,"statfs":{"total":322122547200,"available":322039357440,"internally_reserved":0,"allocated":344064,"data_stored":86517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.786194"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048707+0000","last_change":"2026-02-17T21:10:58.048707+0000","last_active":"2026-02-17T21:10:58.048707+0000","last_peered":"2026-02-17T21:10:58.048707+0000","last_clean":"2026-02-17T21:10:58.048707+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:10:58.048707+0000","last_undegraded":"2026-02-17T21:10:58.048707+0000","last_fullsized":"2026-02-17T21:10:58.048707+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:56:39.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048745+0000","last_change":"2026-02-17T21:10:58.048745+0000","last_active":"2026-02-17T21:10:58.048745+0000","last_peered":"2026-02-17T21:10:58.048745+0000","last_clean":"2026-02-17T21:10:58.048745+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:10:58.048745+0000","last_undegraded":"2026-02-17T21:10:58.048745+0000","last_fullsized":"2026-02-17T21:10:58.048745+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T06:49:23.558997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.403740+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:10:58.403740+0000","last_peered":"2026-02-17T21:10:58.403740+0000","last_clean":"2026-02-17T21:10:58.403740+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:10:58.403740+0000","last_undegraded":"2026-02-17T21:10:58.403740+0000","last_fullsized":"2026-02-17T21:10:58.403740+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:35:26.275365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048849+0000","last_change":"2026-02-17T21:10:58.048849+0000","last_active":"2026-02-17T21:10:58.048849+0000","last_peered":"2026-02-17T21:10:58.048849+0000","last_clean":"2026-02-17T21:10:58.048849+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:10:58.048849+0000","last_undegraded":"2026-02-17T21:10:58.048849+0000","last_fullsized":"2026-02-17T21:10:58.048849+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:21:32.893769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048464+0000","last_change":"2026-02-17T21:10:58.048464+0000","last_active":"2026-02-17T21:10:58.048464+0000","last_peered":"2026-02-17T21:10:58.048464+0000","last_clean":"2026-02-17T21:10:58.048464+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:10:58.048464+0000","last_undegraded":"2026-02-17T21:10:58.048464+0000","last_fullsized":"2026-02-17T21:10:58.048464+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T08:31:58.499464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.049445+0000","last_change":"2026-02-17T21:10:58.049445+0000","last_active":"2026-02-17T21:10:58.049445+0000","last_peered":"2026-02-17T21:10:58.049445+0000","last_clean":"2026-02-17T21:10:58.049445+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:10:58.049445+0000","last_undegraded":"2026-02-17T21:10:58.049445+0000","last_fullsized":"2026-02-17T21:10:58.049445+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T07:53:55.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.403767+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:10:58.403767+0000","last_peered":"2026-02-17T21:10:58.403767+0000","last_clean":"2026-02-17T21:10:58.403767+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:10:58.403767+0000","last_undegraded":"2026-02-17T21:10:58.403767+0000","last_fullsized":"2026-02-17T21:10:58.403767+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T02:30:55.687423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048469+0000","last_change":"2026-02-17T21:10:58.048469+0000","last_active":"2026-02-17T21:10:58.048469+0000","last_peered":"2026-02-17T21:10:58.048469+0000","last_clean":"2026-02-17T21:10:58.048469+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:10:58.048469+0000","last_undegraded":"2026-02-17T21:10:58.048469+0000","last_fullsized":"2026-02-17T21:10:58.048469+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:08:41.667195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":118784,"data_stored":29907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":114688,"data_stored":29888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":108,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":110592,"data_stored":26722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:11:00.291 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:11:00.448 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:00.449 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:11:00.457 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-17T21:10:58.833472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81240,"kb_used_data":336,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491560,"statfs":{"total":322122547200,"available":322039357440,"internally_reserved":0,"allocated":344064,"data_stored":86517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.786194"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048707+0000","last_change":"2026-02-17T21:10:58.048707+0000","last_active":"2026-02-17T21:10:58.048707+0000","last_peered":"2026-02-17T21:10:58.048707+0000","last_clean":"2026-02-17T21:10:58.048707+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:10:58.048707+0000","last_undegraded":"2026-02-17T21:10:58.048707+0000","last_fullsized":"2026-02-17T21:10:58.048707+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:56:39.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048745+0000","last_change":"2026-02-17T21:10:58.048745+0000","last_active":"2026-02-17T21:10:58.048745+0000","last_peered":"2026-02-17T21:10:58.048745+0000","last_clean":"2026-02-17T21:10:58.048745+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:10:58.048745+0000","last_undegraded":"2026-02-17T21:10:58.048745+0000","last_fullsized":"2026-02-17T21:10:58.048745+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T06:49:23.558997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.403740+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:10:58.403740+0000","last_peered":"2026-02-17T21:10:58.403740+0000","last_clean":"2026-02-17T21:10:58.403740+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:10:58.403740+0000","last_undegraded":"2026-02-17T21:10:58.403740+0000","last_fullsized":"2026-02-17T21:10:58.403740+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:35:26.275365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048849+0000","last_change":"2026-02-17T21:10:58.048849+0000","last_active":"2026-02-17T21:10:58.048849+0000","last_peered":"2026-02-17T21:10:58.048849+0000","last_clean":"2026-02-17T21:10:58.048849+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:10:58.048849+0000","last_undegraded":"2026-02-17T21:10:58.048849+0000","last_fullsized":"2026-02-17T21:10:58.048849+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:21:32.893769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048464+0000","last_change":"2026-02-17T21:10:58.048464+0000","last_active":"2026-02-17T21:10:58.048464+0000","last_peered":"2026-02-17T21:10:58.048464+0000","last_clean":"2026-02-17T21:10:58.048464+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:10:58.048464+0000","last_undegraded":"2026-02-17T21:10:58.048464+0000","last_fullsized":"2026-02-17T21:10:58.048464+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T08:31:58.499464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.049445+0000","last_change":"2026-02-17T21:10:58.049445+0000","last_active":"2026-02-17T21:10:58.049445+0000","last_peered":"2026-02-17T21:10:58.049445+0000","last_clean":"2026-02-17T21:10:58.049445+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:10:58.049445+0000","last_undegraded":"2026-02-17T21:10:58.049445+0000","last_fullsized":"2026-02-17T21:10:58.049445+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T07:53:55.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.403767+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:10:58.403767+0000","last_peered":"2026-02-17T21:10:58.403767+0000","last_clean":"2026-02-17T21:10:58.403767+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:10:58.403767+0000","last_undegraded":"2026-02-17T21:10:58.403767+0000","last_fullsized":"2026-02-17T21:10:58.403767+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T02:30:55.687423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-17T21:10:58.048469+0000","last_change":"2026-02-17T21:10:58.048469+0000","last_active":"2026-02-17T21:10:58.048469+0000","last_peered":"2026-02-17T21:10:58.048469+0000","last_clean":"2026-02-17T21:10:58.048469+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:10:58.048469+0000","last_undegraded":"2026-02-17T21:10:58.048469+0000","last_fullsized":"2026-02-17T21:10:58.048469+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:08:41.667195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":118784,"data_stored":29907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":114688,"data_stored":29888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":108,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":110592,"data_stored":26722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:11:00.458 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T21:11:00.459 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-17T21:11:00.459 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-17T21:11:00.459 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T21:11:00.631 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:00.631 INFO:teuthology.orchestra.run.trial060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-17T21:11:00.640 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-17T21:11:00.640 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T21:11:00.665 INFO:tasks.workunit:Pulling workunits from ref d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:11:00.666 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T21:11:00.666 INFO:tasks.workunit:timeout=3h 2026-02-17T21:11:00.666 INFO:tasks.workunit:cleanup=True 2026-02-17T21:11:00.667 DEBUG:teuthology.orchestra.run.trial060:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T21:11:00.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:11:00.683 INFO:teuthology.orchestra.run.trial060.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T21:11:00.683 DEBUG:teuthology.orchestra.run.trial060:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T21:11:00.740 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T21:11:00.740 DEBUG:teuthology.orchestra.run.trial060:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T21:11:00.796 DEBUG:teuthology.orchestra.run.trial060:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:11:00.860 INFO:tasks.workunit.client.0.trial060.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T21:11:21.186 INFO:tasks.workunit.client.0.trial060.stderr:Note: switching to 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'. 2026-02-17T21:11:21.186 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.186 INFO:tasks.workunit.client.0.trial060.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T21:11:21.186 INFO:tasks.workunit.client.0.trial060.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T21:11:21.186 INFO:tasks.workunit.client.0.trial060.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: git switch -c 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr:Or undo this operation with: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: git switch - 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T21:11:21.187 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:21.188 INFO:tasks.workunit.client.0.trial060.stderr:HEAD is now at d855f53b89f Merge branch 'wip-74104-squid' of https://github.com/cbodley/ceph into wip-yuri6-testing-2026-02-17-1732-squid 2026-02-17T21:11:21.193 DEBUG:teuthology.orchestra.run.trial060:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T21:11:21.251 INFO:tasks.workunit.client.0.trial060.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-17T21:11:21.252 INFO:tasks.workunit.client.0.trial060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-17T21:11:21.253 INFO:tasks.workunit.client.0.trial060.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-17T21:11:21.340 INFO:tasks.workunit.client.0.trial060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-17T21:11:21.370 INFO:tasks.workunit.client.0.trial060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-17T21:11:21.395 INFO:tasks.workunit.client.0.trial060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-17T21:11:21.396 INFO:tasks.workunit.client.0.trial060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-17T21:11:21.396 INFO:tasks.workunit.client.0.trial060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-17T21:11:21.420 INFO:tasks.workunit.client.0.trial060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-17T21:11:21.423 DEBUG:teuthology.orchestra.run.trial060:> set -ex 2026-02-17T21:11:21.423 DEBUG:teuthology.orchestra.run.trial060:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T21:11:21.481 INFO:tasks.workunit:Running workunits matching cephtool/test_kvstore_tool.sh on client.0... 2026-02-17T21:11:21.483 INFO:tasks.workunit:Running workunit cephtool/test_kvstore_tool.sh... 2026-02-17T21:11:21.484 DEBUG:teuthology.orchestra.run.trial060:workunit test cephtool/test_kvstore_tool.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh 2026-02-17T21:11:21.541 INFO:tasks.workunit.client.0.trial060.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh 2026-02-17T21:11:21.544 INFO:tasks.workunit.client.0.trial060.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/../../standalone/ceph-helpers.sh 2026-02-17T21:11:21.544 INFO:tasks.workunit.client.0.trial060.stderr:++ TIMEOUT=300 2026-02-17T21:11:21.544 INFO:tasks.workunit.client.0.trial060.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-17T21:11:21.544 INFO:tasks.workunit.client.0.trial060.stderr:++ MAX_TIMEOUT=15 2026-02-17T21:11:21.544 INFO:tasks.workunit.client.0.trial060.stderr:++ PG_NUM=4 2026-02-17T21:11:21.544 INFO:tasks.workunit.client.0.trial060.stderr:++ TMPDIR=/tmp 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ type xmlstarlet 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ XMLSTARLET=xmlstarlet 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:+++ uname 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ '[' Linux = FreeBSD ']' 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ SED=sed 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:++ AWK=awk 2026-02-17T21:11:21.545 INFO:tasks.workunit.client.0.trial060.stderr:+++ stty -a 2026-02-17T21:11:21.546 INFO:tasks.workunit.client.0.trial060.stderr:+++ head -1 2026-02-17T21:11:21.546 INFO:tasks.workunit.client.0.trial060.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-17T21:11:21.547 INFO:tasks.workunit.client.0.trial060.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T21:11:21.547 INFO:tasks.workunit.client.0.trial060.stderr:++ termwidth= 2026-02-17T21:11:21.547 INFO:tasks.workunit.client.0.trial060.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-17T21:11:21.547 INFO:tasks.workunit.client.0.trial060.stderr:++ DIFFCOLOPTS='-y ' 2026-02-17T21:11:21.547 INFO:tasks.workunit.client.0.trial060.stderr:++ KERNCORE=kernel.core_pattern 2026-02-17T21:11:21.547 INFO:tasks.workunit.client.0.trial060.stderr:++ EXTRA_OPTS= 2026-02-17T21:11:21.548 INFO:tasks.workunit.client.0.trial060.stderr:++ test '' = TESTS 2026-02-17T21:11:21.548 INFO:tasks.workunit.client.0.trial060.stderr:+ set -e 2026-02-17T21:11:21.548 INFO:tasks.workunit.client.0.trial060.stderr:+ set -o functrace 2026-02-17T21:11:21.549 INFO:tasks.workunit.client.0.trial060.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T21:11:21.549 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:10: : SUDO=sudo 2026-02-17T21:11:21.549 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:11: : export CEPH_DEV=1 2026-02-17T21:11:21.549 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:11: : CEPH_DEV=1 2026-02-17T21:11:21.549 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:13: : echo note: test ceph_kvstore_tool with bluestore 2026-02-17T21:11:21.549 INFO:tasks.workunit.client.0.trial060.stdout:note: test ceph_kvstore_tool with bluestore 2026-02-17T21:11:21.550 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:21: : mktemp -d ./cephtool.XXX 2026-02-17T21:11:21.550 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:21: : TEMP_DIR=./cephtool.NM8 2026-02-17T21:11:21.550 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:22: : trap 'rm -fr ./cephtool.NM8' 0 2026-02-17T21:11:21.550 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:24: : mktemp ./cephtool.NM8/test_invalid.XXX 2026-02-17T21:11:21.551 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:24: : TEMP_FILE=./cephtool.NM8/test_invalid.no5 2026-02-17T21:11:21.551 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:69: : test_ceph_kvstore_tool 2026-02-17T21:11:21.551 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:29: test_ceph_kvstore_tool: ceph-objectstore-tool --data-path ./cephtool.NM8 --op mkfs --no-mon-config 2026-02-17T21:11:23.111 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:32: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 list 2026-02-17T21:11:23.111 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:32: test_ceph_kvstore_tool: wc -l 2026-02-17T21:11:24.151 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:32: test_ceph_kvstore_tool: origin_kv_nums=11 2026-02-17T21:11:24.152 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:35: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 list 2026-02-17T21:11:24.152 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:35: test_ceph_kvstore_tool: head -n 1 2026-02-17T21:11:24.152 INFO:tasks.workunit.client.0.trial060.stderr://home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:35: test_ceph_kvstore_tool: awk '{print $1}' 2026-02-17T21:11:25.197 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:35: test_ceph_kvstore_tool: prefix=B 2026-02-17T21:11:25.197 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:36: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 exists B 2026-02-17T21:11:25.211 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.209+0000 7f36e3151a80 1 bdev(0x563ccda01c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:25.211 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.209+0000 7f36e3151a80 0 bdev(0x563ccda01c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:25.211 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.209+0000 7f36e3151a80 1 bdev(0x563ccda01c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:25.212 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.210+0000 7f36e3151a80 1 bdev(0x563ccda01c00 ./cephtool.NM8/block) close 2026-02-17T21:11:25.473 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bdev(0x563ccda01c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:25.473 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 0 bdev(0x563ccda01c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bdev(0x563ccda01c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bdev(0x563ccdb80000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 0 bdev(0x563ccdb80000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bdev(0x563ccdb80000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bluefs mount 2026-02-17T21:11:25.474 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.471+0000 7f36e3151a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:25.475 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.475 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.475 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.476 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.476 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.476 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:25.476 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.473+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:25.481 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.479+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:25.481 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.479+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:25.481 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.479+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:25.481 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.479+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:25.481 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.479+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:25.482 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 freelist init 2026-02-17T21:11:25.482 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 freelist _read_cfg 2026-02-17T21:11:25.482 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-17T21:11:25.482 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 0 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-17T21:11:25.483 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000114 seconds, extent_count=1 2026-02-17T21:11:25.483 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:25.483 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 bluefs umount 2026-02-17T21:11:25.483 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.480+0000 7f36e3151a80 1 bdev(0x563ccdb80000 ./cephtool.NM8/block) close 2026-02-17T21:11:25.739 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.737+0000 7f36e3151a80 1 bdev(0x563ccdb80000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:25.739 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.737+0000 7f36e3151a80 0 bdev(0x563ccdb80000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:25.739 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.737+0000 7f36e3151a80 1 bdev(0x563ccdb80000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:25.739 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.737+0000 7f36e3151a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:25.739 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.737+0000 7f36e3151a80 1 bluefs mount 2026-02-17T21:11:25.739 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.737+0000 7f36e3151a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluefs mount shared_bdev_used = 4653056 2026-02-17T21:11:25.740 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.738+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:25.744 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.743+0000 7f36e3151a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:25.745 INFO:tasks.workunit.client.0.trial060.stdout:(B, ) exists 2026-02-17T21:11:25.745 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.743+0000 7f36e3151a80 1 bluefs umount 2026-02-17T21:11:25.745 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:25.743+0000 7f36e3151a80 1 bdev(0x563ccdb80000 ./cephtool.NM8/block) close 2026-02-17T21:11:26.001 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.000+0000 7f36e3151a80 1 freelist shutdown 2026-02-17T21:11:26.001 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.000+0000 7f36e3151a80 1 bdev(0x563ccda01c00 ./cephtool.NM8/block) close 2026-02-17T21:11:26.240 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:37: test_ceph_kvstore_tool: expect_false ceph-kvstore-tool bluestore-kv ./cephtool.NM8 exists Bnotexist 2026-02-17T21:11:26.241 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:17: expect_false: set -x 2026-02-17T21:11:26.241 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:18: expect_false: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 exists Bnotexist 2026-02-17T21:11:26.256 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.254+0000 7fb22780da80 1 bdev(0x55ad030bdc00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:26.256 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.254+0000 7fb22780da80 0 bdev(0x55ad030bdc00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:26.256 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.254+0000 7fb22780da80 1 bdev(0x55ad030bdc00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:26.257 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.254+0000 7fb22780da80 1 bdev(0x55ad030bdc00 ./cephtool.NM8/block) close 2026-02-17T21:11:26.518 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bdev(0x55ad030bdc00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:26.518 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 0 bdev(0x55ad030bdc00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bdev(0x55ad030bdc00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bdev(0x55ad0323c000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 0 bdev(0x55ad0323c000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bdev(0x55ad0323c000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bluefs mount 2026-02-17T21:11:26.519 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.516+0000 7fb22780da80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:26.520 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.520 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.521 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.521 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.521 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.521 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:26.521 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.518+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:26.526 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.524+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:26.526 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.524+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:26.526 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.524+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:26.526 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.524+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:26.526 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.524+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:26.527 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 freelist init 2026-02-17T21:11:26.527 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 freelist _read_cfg 2026-02-17T21:11:26.527 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-17T21:11:26.528 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 0 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-17T21:11:26.528 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000110 seconds, extent_count=1 2026-02-17T21:11:26.528 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:26.528 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 bluefs umount 2026-02-17T21:11:26.528 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.525+0000 7fb22780da80 1 bdev(0x55ad0323c000 ./cephtool.NM8/block) close 2026-02-17T21:11:26.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.778+0000 7fb22780da80 1 bdev(0x55ad0323c000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:26.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.778+0000 7fb22780da80 0 bdev(0x55ad0323c000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:26.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.778+0000 7fb22780da80 1 bdev(0x55ad0323c000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:26.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.778+0000 7fb22780da80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:26.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.778+0000 7fb22780da80 1 bluefs mount 2026-02-17T21:11:26.781 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.778+0000 7fb22780da80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:26.781 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.781 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.781 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.782 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.782 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:26.782 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluefs mount shared_bdev_used = 4653056 2026-02-17T21:11:26.782 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.779+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:26.785 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.784+0000 7fb22780da80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:26.786 INFO:tasks.workunit.client.0.trial060.stdout:(Bnotexist, ) does not exist 2026-02-17T21:11:26.786 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.784+0000 7fb22780da80 1 bluefs umount 2026-02-17T21:11:26.786 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:26.784+0000 7fb22780da80 1 bdev(0x55ad0323c000 ./cephtool.NM8/block) close 2026-02-17T21:11:27.041 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.040+0000 7fb22780da80 1 freelist shutdown 2026-02-17T21:11:27.042 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.040+0000 7fb22780da80 1 bdev(0x55ad030bdc00 ./cephtool.NM8/block) close 2026-02-17T21:11:27.283 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:18: expect_false: return 0 2026-02-17T21:11:27.284 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:40: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 list-crc 2026-02-17T21:11:27.297 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.295+0000 7f27ec7d3a80 1 bdev(0x561a23059c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:27.297 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.295+0000 7f27ec7d3a80 0 bdev(0x561a23059c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:27.297 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.295+0000 7f27ec7d3a80 1 bdev(0x561a23059c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:27.298 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.296+0000 7f27ec7d3a80 1 bdev(0x561a23059c00 ./cephtool.NM8/block) close 2026-02-17T21:11:27.560 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bdev(0x561a23059c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 0 bdev(0x561a23059c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bdev(0x561a23059c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bdev(0x561a231d8000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 0 bdev(0x561a231d8000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bdev(0x561a231d8000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bluefs mount 2026-02-17T21:11:27.561 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.558+0000 7f27ec7d3a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:27.562 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.562 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.562 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.563 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.563 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.563 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:27.563 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.560+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:27.568 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.566+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:27.568 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.566+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:27.568 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.566+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:27.568 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.566+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:27.568 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.566+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:27.569 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 freelist init 2026-02-17T21:11:27.569 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 freelist _read_cfg 2026-02-17T21:11:27.569 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-17T21:11:27.570 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 0 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-17T21:11:27.570 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000112 seconds, extent_count=1 2026-02-17T21:11:27.570 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:27.570 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 bluefs umount 2026-02-17T21:11:27.570 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.567+0000 7f27ec7d3a80 1 bdev(0x561a231d8000 ./cephtool.NM8/block) close 2026-02-17T21:11:27.822 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bdev(0x561a231d8000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:27.822 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 0 bdev(0x561a231d8000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bdev(0x561a231d8000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluefs mount shared_bdev_used = 4653056 2026-02-17T21:11:27.823 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.820+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:27.827 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.825+0000 7f27ec7d3a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:B blocks 3887034139 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:B blocks_per_key 2356384685 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:B bytes_per_block 1745006751 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:B size 714193852 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:S blobid_max 3417961130 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:S freelist_type 753695201 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:S min_alloc_size 1458662255 2026-02-17T21:11:27.828 INFO:tasks.workunit.client.0.trial060.stdout:S min_compat_ondisk_format 2796427507 2026-02-17T21:11:27.829 INFO:tasks.workunit.client.0.trial060.stdout:S nid_max 1692495819 2026-02-17T21:11:27.829 INFO:tasks.workunit.client.0.trial060.stdout:S ondisk_format 3256356326 2026-02-17T21:11:27.829 INFO:tasks.workunit.client.0.trial060.stdout:S per_pool_omap 490065073 2026-02-17T21:11:27.829 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.826+0000 7f27ec7d3a80 1 bluefs umount 2026-02-17T21:11:27.829 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:27.826+0000 7f27ec7d3a80 1 bdev(0x561a231d8000 ./cephtool.NM8/block) close 2026-02-17T21:11:28.083 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.082+0000 7f27ec7d3a80 1 freelist shutdown 2026-02-17T21:11:28.084 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.082+0000 7f27ec7d3a80 1 bdev(0x561a23059c00 ./cephtool.NM8/block) close 2026-02-17T21:11:28.323 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:41: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 list-crc B 2026-02-17T21:11:28.337 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.335+0000 7f3f64e2ca80 1 bdev(0x5615a5211c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:28.337 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.335+0000 7f3f64e2ca80 0 bdev(0x5615a5211c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:28.338 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.336+0000 7f3f64e2ca80 1 bdev(0x5615a5211c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:28.338 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.336+0000 7f3f64e2ca80 1 bdev(0x5615a5211c00 ./cephtool.NM8/block) close 2026-02-17T21:11:28.600 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bdev(0x5615a5211c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:28.600 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 0 bdev(0x5615a5211c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bdev(0x5615a5211c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bdev(0x5615a5390000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 0 bdev(0x5615a5390000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bdev(0x5615a5390000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bluefs mount 2026-02-17T21:11:28.601 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.598+0000 7f3f64e2ca80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:28.602 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.602 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.602 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.602 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.602 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.602 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:28.603 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.600+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:28.607 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:28.608 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:28.608 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:28.608 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:28.608 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 freelist init 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 freelist _read_cfg 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.606+0000 7f3f64e2ca80 0 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.607+0000 7f3f64e2ca80 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000119 seconds, extent_count=1 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.607+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.607+0000 7f3f64e2ca80 1 bluefs umount 2026-02-17T21:11:28.609 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.607+0000 7f3f64e2ca80 1 bdev(0x5615a5390000 ./cephtool.NM8/block) close 2026-02-17T21:11:28.865 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.863+0000 7f3f64e2ca80 1 bdev(0x5615a5390000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:28.865 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.863+0000 7f3f64e2ca80 0 bdev(0x5615a5390000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:28.865 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.863+0000 7f3f64e2ca80 1 bdev(0x5615a5390000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:28.865 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.863+0000 7f3f64e2ca80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:28.865 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.863+0000 7f3f64e2ca80 1 bluefs mount 2026-02-17T21:11:28.865 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.863+0000 7f3f64e2ca80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluefs mount shared_bdev_used = 4653056 2026-02-17T21:11:28.866 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.864+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:28.870 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.869+0000 7f3f64e2ca80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:28.871 INFO:tasks.workunit.client.0.trial060.stdout:B blocks 3887034139 2026-02-17T21:11:28.871 INFO:tasks.workunit.client.0.trial060.stdout:B blocks_per_key 2356384685 2026-02-17T21:11:28.871 INFO:tasks.workunit.client.0.trial060.stdout:B bytes_per_block 1745006751 2026-02-17T21:11:28.871 INFO:tasks.workunit.client.0.trial060.stdout:B size 714193852 2026-02-17T21:11:28.871 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.869+0000 7f3f64e2ca80 1 bluefs umount 2026-02-17T21:11:28.872 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:28.869+0000 7f3f64e2ca80 1 bdev(0x5615a5390000 ./cephtool.NM8/block) close 2026-02-17T21:11:29.126 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.125+0000 7f3f64e2ca80 1 freelist shutdown 2026-02-17T21:11:29.127 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.125+0000 7f3f64e2ca80 1 bdev(0x5615a5211c00 ./cephtool.NM8/block) close 2026-02-17T21:11:29.365 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:44: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 list B 2026-02-17T21:11:29.379 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.377+0000 7f1065bcca80 1 bdev(0x5559ed591c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:29.379 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.377+0000 7f1065bcca80 0 bdev(0x5559ed591c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:29.379 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.377+0000 7f1065bcca80 1 bdev(0x5559ed591c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:29.379 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.377+0000 7f1065bcca80 1 bdev(0x5559ed591c00 ./cephtool.NM8/block) close 2026-02-17T21:11:29.644 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bdev(0x5559ed591c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 0 bdev(0x5559ed591c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bdev(0x5559ed591c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bdev(0x5559ed710000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 0 bdev(0x5559ed710000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bdev(0x5559ed710000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bluefs mount 2026-02-17T21:11:29.645 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.642+0000 7f1065bcca80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:29.646 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.646 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.646 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.647 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.647 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.647 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:29.647 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.644+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:29.651 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.649+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:29.651 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.649+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:29.651 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.649+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:29.652 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.649+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:29.652 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.649+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:29.652 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 freelist init 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 freelist _read_cfg 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 0 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000123 seconds, extent_count=1 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 bluefs umount 2026-02-17T21:11:29.653 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.650+0000 7f1065bcca80 1 bdev(0x5559ed710000 ./cephtool.NM8/block) close 2026-02-17T21:11:29.906 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.904+0000 7f1065bcca80 1 bdev(0x5559ed710000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:29.906 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.904+0000 7f1065bcca80 0 bdev(0x5559ed710000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:29.906 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.904+0000 7f1065bcca80 1 bdev(0x5559ed710000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:29.906 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.904+0000 7f1065bcca80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:29.906 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.904+0000 7f1065bcca80 1 bluefs mount 2026-02-17T21:11:29.907 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.904+0000 7f1065bcca80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:29.907 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.907 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.908 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.908 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.908 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:29.908 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluefs mount shared_bdev_used = 4653056 2026-02-17T21:11:29.908 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.905+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:29.912 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.911+0000 7f1065bcca80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:29.913 INFO:tasks.workunit.client.0.trial060.stdout:B blocks 2026-02-17T21:11:29.913 INFO:tasks.workunit.client.0.trial060.stdout:B blocks_per_key 2026-02-17T21:11:29.913 INFO:tasks.workunit.client.0.trial060.stdout:B bytes_per_block 2026-02-17T21:11:29.913 INFO:tasks.workunit.client.0.trial060.stdout:B size 2026-02-17T21:11:29.914 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.911+0000 7f1065bcca80 1 bluefs umount 2026-02-17T21:11:29.914 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:29.911+0000 7f1065bcca80 1 bdev(0x5559ed710000 ./cephtool.NM8/block) close 2026-02-17T21:11:30.167 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.166+0000 7f1065bcca80 1 freelist shutdown 2026-02-17T21:11:30.168 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.166+0000 7f1065bcca80 1 bdev(0x5559ed591c00 ./cephtool.NM8/block) close 2026-02-17T21:11:30.408 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:47: test_ceph_kvstore_tool: echo helloworld 2026-02-17T21:11:30.408 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:48: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 set TESTPREFIX TESTKEY in ./cephtool.NM8/test_invalid.no5 2026-02-17T21:11:30.423 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.421+0000 7fe947d96a80 1 bdev(0x55da6f48fc00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:30.424 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.421+0000 7fe947d96a80 0 bdev(0x55da6f48fc00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:30.424 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.422+0000 7fe947d96a80 1 bdev(0x55da6f48fc00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:30.424 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.422+0000 7fe947d96a80 1 bdev(0x55da6f48fc00 ./cephtool.NM8/block) close 2026-02-17T21:11:30.686 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bdev(0x55da6f48fc00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 0 bdev(0x55da6f48fc00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bdev(0x55da6f48fc00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bdev(0x55da6f60c000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 0 bdev(0x55da6f60c000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bdev(0x55da6f60c000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:30.687 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:30.688 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bluefs mount 2026-02-17T21:11:30.688 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.684+0000 7fe947d96a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:30.688 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.689 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.689 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.689 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.689 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.689 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:30.689 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.686+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.692+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.692+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.692+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.692+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.692+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 freelist init 2026-02-17T21:11:30.694 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 freelist _read_cfg 2026-02-17T21:11:30.695 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-17T21:11:30.695 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 0 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-17T21:11:30.696 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000112 seconds, extent_count=1 2026-02-17T21:11:30.696 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:30.696 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 bluefs umount 2026-02-17T21:11:30.696 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.693+0000 7fe947d96a80 1 bdev(0x55da6f60c000 ./cephtool.NM8/block) close 2026-02-17T21:11:30.948 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.946+0000 7fe947d96a80 1 bdev(0x55da6f60c000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:30.948 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.946+0000 7fe947d96a80 0 bdev(0x55da6f60c000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:30.948 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.946+0000 7fe947d96a80 1 bdev(0x55da6f60c000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:30.948 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.946+0000 7fe947d96a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:30.949 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.946+0000 7fe947d96a80 1 bluefs mount 2026-02-17T21:11:30.949 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.946+0000 7fe947d96a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:30.949 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.949 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.949 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.949 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.950 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:30.950 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluefs mount shared_bdev_used = 4653056 2026-02-17T21:11:30.950 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.947+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:30.958 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.956+0000 7fe947d96a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:30.960 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.958+0000 7fe947d96a80 1 bluefs umount 2026-02-17T21:11:30.960 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:30.958+0000 7fe947d96a80 1 bdev(0x55da6f60c000 ./cephtool.NM8/block) close 2026-02-17T21:11:31.225 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.224+0000 7fe947d96a80 1 freelist shutdown 2026-02-17T21:11:31.226 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.224+0000 7fe947d96a80 1 bdev(0x55da6f48fc00 ./cephtool.NM8/block) close 2026-02-17T21:11:31.451 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:49: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 exists TESTPREFIX TESTKEY 2026-02-17T21:11:31.466 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.464+0000 7fb429da4a80 1 bdev(0x55f875be3c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:31.466 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.464+0000 7fb429da4a80 0 bdev(0x55f875be3c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:31.466 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.464+0000 7fb429da4a80 1 bdev(0x55f875be3c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:31.466 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.464+0000 7fb429da4a80 1 bdev(0x55f875be3c00 ./cephtool.NM8/block) close 2026-02-17T21:11:31.728 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bdev(0x55f875be3c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:31.728 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 0 bdev(0x55f875be3c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bdev(0x55f875be3c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bdev(0x55f875d60000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 0 bdev(0x55f875d60000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bdev(0x55f875d60000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bluefs mount 2026-02-17T21:11:31.729 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.727+0000 7fb429da4a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:31.730 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.730 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.730 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.730 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.731 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.731 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:31.731 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.729+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:31.736 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:31.736 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:31.736 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:31.736 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:31.736 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:31.737 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 freelist init 2026-02-17T21:11:31.737 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.735+0000 7fb429da4a80 1 freelist _read_cfg 2026-02-17T21:11:31.737 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.736+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:31.737 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.736+0000 7fb429da4a80 1 bluefs umount 2026-02-17T21:11:31.737 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.736+0000 7fb429da4a80 1 bdev(0x55f875d60000 ./cephtool.NM8/block) close 2026-02-17T21:11:31.990 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.989+0000 7fb429da4a80 1 bdev(0x55f875d60000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:31.990 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.989+0000 7fb429da4a80 0 bdev(0x55f875d60000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:31.990 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.989+0000 7fb429da4a80 1 bdev(0x55f875d60000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:31.990 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.989+0000 7fb429da4a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:31.990 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.989+0000 7fb429da4a80 1 bluefs mount 2026-02-17T21:11:31.991 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.989+0000 7fb429da4a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:31.991 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.991 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.992 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.992 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.992 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:31.992 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluefs mount shared_bdev_used = 4915200 2026-02-17T21:11:31.992 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.990+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:31.996 INFO:tasks.workunit.client.0.trial060.stdout:(TESTPREFIX, TESTKEY) exists 2026-02-17T21:11:31.997 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.996+0000 7fb429da4a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:31.997 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.996+0000 7fb429da4a80 1 bluefs umount 2026-02-17T21:11:31.997 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:31.996+0000 7fb429da4a80 1 bdev(0x55f875d60000 ./cephtool.NM8/block) close 2026-02-17T21:11:32.255 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.255+0000 7fb429da4a80 1 freelist shutdown 2026-02-17T21:11:32.256 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.255+0000 7fb429da4a80 1 bdev(0x55f875be3c00 ./cephtool.NM8/block) close 2026-02-17T21:11:32.491 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:52: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 get TESTPREFIX TESTKEY out ./cephtool.NM8/test_invalid.no5.bak 2026-02-17T21:11:32.507 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.506+0000 7fca1982ba80 1 bdev(0x55baa4361c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:32.508 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.506+0000 7fca1982ba80 0 bdev(0x55baa4361c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:32.508 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.506+0000 7fca1982ba80 1 bdev(0x55baa4361c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:32.508 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.507+0000 7fca1982ba80 1 bdev(0x55baa4361c00 ./cephtool.NM8/block) close 2026-02-17T21:11:32.770 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bdev(0x55baa4361c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 0 bdev(0x55baa4361c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bdev(0x55baa4361c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bdev(0x55baa44de000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 0 bdev(0x55baa44de000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bdev(0x55baa44de000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:32.771 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bluefs mount 2026-02-17T21:11:32.772 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.769+0000 7fca1982ba80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:32.773 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.771+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:32.779 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:32.779 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:32.779 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:32.779 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:32.779 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:32.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 freelist init 2026-02-17T21:11:32.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.778+0000 7fca1982ba80 1 freelist _read_cfg 2026-02-17T21:11:32.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.779+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:32.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.779+0000 7fca1982ba80 1 bluefs umount 2026-02-17T21:11:32.780 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:32.779+0000 7fca1982ba80 1 bdev(0x55baa44de000 ./cephtool.NM8/block) close 2026-02-17T21:11:33.032 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.031+0000 7fca1982ba80 1 bdev(0x55baa44de000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:33.032 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.031+0000 7fca1982ba80 0 bdev(0x55baa44de000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:33.032 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.031+0000 7fca1982ba80 1 bdev(0x55baa44de000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:33.033 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.031+0000 7fca1982ba80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:33.033 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.031+0000 7fca1982ba80 1 bluefs mount 2026-02-17T21:11:33.033 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.031+0000 7fca1982ba80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:33.033 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.034 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.034 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.034 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.034 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.034 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluefs mount shared_bdev_used = 4915200 2026-02-17T21:11:33.034 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.032+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:33.039 INFO:tasks.workunit.client.0.trial060.stdout:(TESTPREFIX, TESTKEY) 2026-02-17T21:11:33.039 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.038+0000 7fca1982ba80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:33.039 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.038+0000 7fca1982ba80 1 bluefs umount 2026-02-17T21:11:33.039 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.038+0000 7fca1982ba80 1 bdev(0x55baa44de000 ./cephtool.NM8/block) close 2026-02-17T21:11:33.293 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.293+0000 7fca1982ba80 1 freelist shutdown 2026-02-17T21:11:33.294 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.293+0000 7fca1982ba80 1 bdev(0x55baa4361c00 ./cephtool.NM8/block) close 2026-02-17T21:11:33.534 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:53: test_ceph_kvstore_tool: diff ./cephtool.NM8/test_invalid.no5 ./cephtool.NM8/test_invalid.no5.bak 2026-02-17T21:11:33.535 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:56: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 rm TESTPREFIX TESTKEY 2026-02-17T21:11:33.550 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.549+0000 7f1daa3c7a80 1 bdev(0x560a82eedc00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:33.550 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.549+0000 7f1daa3c7a80 0 bdev(0x560a82eedc00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:33.550 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.549+0000 7f1daa3c7a80 1 bdev(0x560a82eedc00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:33.550 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.549+0000 7f1daa3c7a80 1 bdev(0x560a82eedc00 ./cephtool.NM8/block) close 2026-02-17T21:11:33.812 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bdev(0x560a82eedc00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:33.812 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 0 bdev(0x560a82eedc00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bdev(0x560a82eedc00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bdev(0x560a8306a000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 0 bdev(0x560a8306a000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bdev(0x560a8306a000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bluefs mount 2026-02-17T21:11:33.813 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.811+0000 7f1daa3c7a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:33.814 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.814 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.814 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.814 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.815 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:33.815 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:33.815 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.813+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:33.820 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:33.820 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:33.820 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:33.820 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:33.820 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:33.820 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 freelist init 2026-02-17T21:11:33.821 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.819+0000 7f1daa3c7a80 1 freelist _read_cfg 2026-02-17T21:11:33.821 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.820+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:33.821 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.820+0000 7f1daa3c7a80 1 bluefs umount 2026-02-17T21:11:33.821 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:33.820+0000 7f1daa3c7a80 1 bdev(0x560a8306a000 ./cephtool.NM8/block) close 2026-02-17T21:11:34.074 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.073+0000 7f1daa3c7a80 1 bdev(0x560a8306a000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:34.074 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.073+0000 7f1daa3c7a80 0 bdev(0x560a8306a000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:34.074 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.073+0000 7f1daa3c7a80 1 bdev(0x560a8306a000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:34.074 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.073+0000 7f1daa3c7a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:34.074 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.073+0000 7f1daa3c7a80 1 bluefs mount 2026-02-17T21:11:34.075 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.073+0000 7f1daa3c7a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:34.075 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.075 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.076 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.076 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.076 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.076 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluefs mount shared_bdev_used = 4915200 2026-02-17T21:11:34.076 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.074+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:34.083 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.083+0000 7f1daa3c7a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:34.086 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.085+0000 7f1daa3c7a80 1 bluefs umount 2026-02-17T21:11:34.086 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.086+0000 7f1daa3c7a80 1 bdev(0x560a8306a000 ./cephtool.NM8/block) close 2026-02-17T21:11:34.349 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.349+0000 7f1daa3c7a80 1 freelist shutdown 2026-02-17T21:11:34.350 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.349+0000 7f1daa3c7a80 1 bdev(0x560a82eedc00 ./cephtool.NM8/block) close 2026-02-17T21:11:34.576 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:57: test_ceph_kvstore_tool: expect_false ceph-kvstore-tool bluestore-kv ./cephtool.NM8 exists TESTPREFIX TESTKEY 2026-02-17T21:11:34.576 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:17: expect_false: set -x 2026-02-17T21:11:34.576 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:18: expect_false: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 exists TESTPREFIX TESTKEY 2026-02-17T21:11:34.591 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.590+0000 7f22c7796a80 1 bdev(0x55a455141c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:34.591 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.590+0000 7f22c7796a80 0 bdev(0x55a455141c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:34.591 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.590+0000 7f22c7796a80 1 bdev(0x55a455141c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:34.591 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.590+0000 7f22c7796a80 1 bdev(0x55a455141c00 ./cephtool.NM8/block) close 2026-02-17T21:11:34.853 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bdev(0x55a455141c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 0 bdev(0x55a455141c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bdev(0x55a455141c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bdev(0x55a4552be000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 0 bdev(0x55a4552be000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bdev(0x55a4552be000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bluefs mount 2026-02-17T21:11:34.854 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.852+0000 7f22c7796a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:34.856 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.855+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:34.861 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.860+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:34.862 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.860+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:34.862 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.860+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:34.862 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.860+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:34.862 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.860+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:34.863 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.861+0000 7f22c7796a80 1 freelist init 2026-02-17T21:11:34.863 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.861+0000 7f22c7796a80 1 freelist _read_cfg 2026-02-17T21:11:34.863 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.862+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:34.863 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.862+0000 7f22c7796a80 1 bluefs umount 2026-02-17T21:11:34.863 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:34.862+0000 7f22c7796a80 1 bdev(0x55a4552be000 ./cephtool.NM8/block) close 2026-02-17T21:11:35.118 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.117+0000 7f22c7796a80 1 bdev(0x55a4552be000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:35.118 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.117+0000 7f22c7796a80 0 bdev(0x55a4552be000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:35.118 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.117+0000 7f22c7796a80 1 bdev(0x55a4552be000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:35.118 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.117+0000 7f22c7796a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:35.118 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.117+0000 7f22c7796a80 1 bluefs mount 2026-02-17T21:11:35.118 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.117+0000 7f22c7796a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:35.119 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.119 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.119 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.119 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.120 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.120 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluefs mount shared_bdev_used = 4980736 2026-02-17T21:11:35.120 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.118+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:35.124 INFO:tasks.workunit.client.0.trial060.stdout:(TESTPREFIX, TESTKEY) does not exist 2026-02-17T21:11:35.125 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.123+0000 7f22c7796a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:35.125 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.124+0000 7f22c7796a80 1 bluefs umount 2026-02-17T21:11:35.125 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.124+0000 7f22c7796a80 1 bdev(0x55a4552be000 ./cephtool.NM8/block) close 2026-02-17T21:11:35.379 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.378+0000 7f22c7796a80 1 freelist shutdown 2026-02-17T21:11:35.380 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.378+0000 7f22c7796a80 1 bdev(0x55a455141c00 ./cephtool.NM8/block) close 2026-02-17T21:11:35.617 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:18: expect_false: return 0 2026-02-17T21:11:35.617 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:60: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 compact 2026-02-17T21:11:35.630 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.628+0000 7ff07141da80 1 bdev(0x564964149c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:35.630 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.628+0000 7ff07141da80 0 bdev(0x564964149c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:35.630 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.628+0000 7ff07141da80 1 bdev(0x564964149c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:35.631 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.628+0000 7ff07141da80 1 bdev(0x564964149c00 ./cephtool.NM8/block) close 2026-02-17T21:11:35.892 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bdev(0x564964149c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 0 bdev(0x564964149c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bdev(0x564964149c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bdev(0x5649642c8000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 0 bdev(0x5649642c8000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bdev(0x5649642c8000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bluefs mount 2026-02-17T21:11:35.893 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.890+0000 7ff07141da80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:35.895 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.895 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.895 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.895 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.895 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:35.896 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:35.896 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.893+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:35.901 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.899+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:35.901 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.899+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:35.901 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.899+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:35.901 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.899+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:35.901 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.899+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:35.902 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.900+0000 7ff07141da80 1 freelist init 2026-02-17T21:11:35.902 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.900+0000 7ff07141da80 1 freelist _read_cfg 2026-02-17T21:11:35.903 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.901+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:35.903 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.901+0000 7ff07141da80 1 bluefs umount 2026-02-17T21:11:35.903 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:35.901+0000 7ff07141da80 1 bdev(0x5649642c8000 ./cephtool.NM8/block) close 2026-02-17T21:11:36.157 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.155+0000 7ff07141da80 1 bdev(0x5649642c8000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:36.157 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.155+0000 7ff07141da80 0 bdev(0x5649642c8000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:36.157 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.155+0000 7ff07141da80 1 bdev(0x5649642c8000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:36.157 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.155+0000 7ff07141da80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:36.158 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.155+0000 7ff07141da80 1 bluefs mount 2026-02-17T21:11:36.158 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.155+0000 7ff07141da80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:36.158 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.159 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.159 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.159 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.159 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.159 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluefs mount shared_bdev_used = 4980736 2026-02-17T21:11:36.159 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.156+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:36.167 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.166+0000 7ff07141da80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:36.168 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.166+0000 7ff07141da80 2 rocksdb: compact starting 2026-02-17T21:11:36.170 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.168+0000 7ff07141da80 2 rocksdb: compact completed 2026-02-17T21:11:36.172 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.170+0000 7ff07141da80 1 bluefs umount 2026-02-17T21:11:36.172 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.170+0000 7ff07141da80 1 bdev(0x5649642c8000 ./cephtool.NM8/block) close 2026-02-17T21:11:36.436 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.435+0000 7ff07141da80 1 freelist shutdown 2026-02-17T21:11:36.437 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.435+0000 7ff07141da80 1 bdev(0x564964149c00 ./cephtool.NM8/block) close 2026-02-17T21:11:36.655 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:63: test_ceph_kvstore_tool: ceph-kvstore-tool bluestore-kv ./cephtool.NM8 destructive-repair 2026-02-17T21:11:36.670 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.667+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:36.670 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.667+0000 7fb8747d0a80 0 bdev(0x55801f031c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:36.670 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.668+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:36.670 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.668+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) close 2026-02-17T21:11:36.932 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 0 bdev(0x55801f031c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 0 bdev(0x55801f1b0000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs mount 2026-02-17T21:11:36.933 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:36.935 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:36.941 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:36.941 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:36.941 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:36.941 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:36.941 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:36.942 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 freelist init 2026-02-17T21:11:36.942 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 freelist _read_cfg 2026-02-17T21:11:36.942 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:36.943 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bluefs umount 2026-02-17T21:11:36.943 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) close 2026-02-17T21:11:37.198 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.198 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.196+0000 7fb8747d0a80 0 bdev(0x55801f1b0000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.198 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.198 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:37.199 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs mount 2026-02-17T21:11:37.199 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:37.199 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.199 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.199 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.200 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.200 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.200 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount shared_bdev_used = 4915200 2026-02-17T21:11:37.200 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:37.219 INFO:tasks.workunit.client.0.trial060.stdout:destructive-repair completed without reporting an error 2026-02-17T21:11:37.229 INFO:tasks.workunit.client.0.trial060.stderr:*** Caught signal (Segmentation fault) ** 2026-02-17T21:11:37.229 INFO:tasks.workunit.client.0.trial060.stderr: in thread 7fb8747d0a80 thread_name:ceph-kvstore-to 2026-02-17T21:11:37.229 INFO:tasks.workunit.client.0.trial060.stderr: ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable) 2026-02-17T21:11:37.229 INFO:tasks.workunit.client.0.trial060.stderr: 1: /lib64/libc.so.6(+0x3fc30) [0x7fb87483fc30] 2026-02-17T21:11:37.229 INFO:tasks.workunit.client.0.trial060.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xb4) [0x55801d26ce54] 2026-02-17T21:11:37.229 INFO:tasks.workunit.client.0.trial060.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string, std::allocator > const&, unsigned int, KeyValueDB::IteratorBounds)+0x240) [0x55801d26d570] 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 4: (BlueStore::_close_db()+0x380) [0x55801d0aa470] 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 5: (BlueStore::umount()+0x244) [0x55801d0b6fe4] 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 6: ceph-kvstore-tool(+0x1f7503) [0x55801d054503] 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 7: main() 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 8: /lib64/libc.so.6(+0x2a610) [0x7fb87482a610] 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 9: __libc_start_main() 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 10: _start() 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr:2026-02-17T21:11:37.217+0000 7fb8747d0a80 -1 *** Caught signal (Segmentation fault) ** 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: in thread 7fb8747d0a80 thread_name:ceph-kvstore-to 2026-02-17T21:11:37.230 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable) 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 1: /lib64/libc.so.6(+0x3fc30) [0x7fb87483fc30] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xb4) [0x55801d26ce54] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string, std::allocator > const&, unsigned int, KeyValueDB::IteratorBounds)+0x240) [0x55801d26d570] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 4: (BlueStore::_close_db()+0x380) [0x55801d0aa470] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 5: (BlueStore::umount()+0x244) [0x55801d0b6fe4] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 6: ceph-kvstore-tool(+0x1f7503) [0x55801d054503] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 7: main() 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 8: /lib64/libc.so.6(+0x2a610) [0x7fb87482a610] 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 9: __libc_start_main() 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: 10: _start() 2026-02-17T21:11:37.231 INFO:tasks.workunit.client.0.trial060.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr:--- begin dump of recent events --- 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -194> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command assert hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -193> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command abort hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -192> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command leak_some_memory hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -191> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perfcounters_dump hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -190> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command 1 hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -189> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf dump hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -188> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perfcounters_schema hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -187> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf histogram dump hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -186> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command 2 hook 0x55801f0249c0 2026-02-17T21:11:37.232 INFO:tasks.workunit.client.0.trial060.stderr: -185> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf schema hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -184> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command counter dump hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -183> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command counter schema hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -182> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf histogram schema hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -181> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf reset hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -180> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config show hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -179> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config help hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -178> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config set hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -177> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config unset hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -176> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config get hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -175> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config diff hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -174> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config diff get hook 0x55801f0249c0 2026-02-17T21:11:37.233 INFO:tasks.workunit.client.0.trial060.stderr: -173> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command injectargs hook 0x55801f0249c0 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -172> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command log flush hook 0x55801f0249c0 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -171> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command log dump hook 0x55801f0249c0 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -170> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command log reopen hook 0x55801f0249c0 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -169> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command dump_mempools hook 0x55801f104068 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -168> 2026-02-17T21:11:36.667+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _open_db_and_around::NCB::read_only=0, to_repair=1 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -167> 2026-02-17T21:11:36.667+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -166> 2026-02-17T21:11:36.667+0000 7fb8747d0a80 0 bdev(0x55801f031c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -165> 2026-02-17T21:11:36.668+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -164> 2026-02-17T21:11:36.668+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) close 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -163> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -162> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 0 bdev(0x55801f031c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -161> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.234 INFO:tasks.workunit.client.0.trial060.stderr: -160> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -159> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore bluefs device info hook 0x55801f0249e0 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -158> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs stats hook 0x55801f0249e0 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -157> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs files list hook 0x55801f0249e0 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -156> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs debug_inject_read_zeros hook 0x55801f0249e0 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -155> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -154> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 0 bdev(0x55801f1b0000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -153> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -152> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -151> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs mount 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -150> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 bluefs mount super: super(uuid fcbec683-5f94-4994-8bd5-75eefc73e648 osd 3d806266-f149-4138-830d-01af17cfcc64 seq 4 block_size 0x1000 log_fnode 0xfile(ino 1 size 0x0 mtime 2026-02-17T21:11:36.170492+0000 allocated 10000 alloc_commit 10000 extents [1:0xc2000~10000])) 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -149> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:37.235 INFO:tasks.workunit.client.0.trial060.stderr: -148> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -147> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -146> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -145> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -144> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -143> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -142> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -141> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -140> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -139> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -138> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -137> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -136> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.236 INFO:tasks.workunit.client.0.trial060.stderr: -135> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -134> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -133> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -132> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -131> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -130> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -129> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -128> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -127> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -126> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -125> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -124> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -123> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.237 INFO:tasks.workunit.client.0.trial060.stderr: -122> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -121> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -120> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -119> 2026-02-17T21:11:36.934+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -118> 2026-02-17T21:11:36.934+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -117> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -116> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -115> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -114> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _open_super_meta::NCB::freelist_type=null 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -113> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -112> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -111> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0 2026-02-17T21:11:37.238 INFO:tasks.workunit.client.0.trial060.stderr: -110> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor snappy 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -109> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor snappy 0x55801f1af530 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -108> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0x55801f1af530 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -107> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor snappy loaded and registered 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -106> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0x55801f1af530 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -105> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zlib = 0 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -104> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor zlib 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -103> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor zlib 0x55801f1af6e0 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -102> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zlib = 0x55801f1af6e0 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -101> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor zlib loaded and registered 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -100> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zlib = 0x55801f1af6e0 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -99> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zstd = 0 2026-02-17T21:11:37.239 INFO:tasks.workunit.client.0.trial060.stderr: -98> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor zstd 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -97> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor zstd 0x55801f1af770 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -96> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zstd = 0x55801f1af770 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -95> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor zstd loaded and registered 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -94> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zstd = 0x55801f1af770 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -93> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor lz4 = 0 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -92> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor lz4 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -91> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor lz4 0x55801f1afcb0 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -90> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor lz4 = 0x55801f1afcb0 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -89> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor lz4 loaded and registered 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -88> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor lz4 = 0x55801f1afcb0 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -87> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0x55801f1af530 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -86> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _open_fm::NCB::freelist_type=null 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -85> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 freelist init 2026-02-17T21:11:37.240 INFO:tasks.workunit.client.0.trial060.stderr: -84> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 freelist _read_cfg 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -83> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator dump block hook 0x55801f1afc80 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -82> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator score block hook 0x55801f1afc80 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -81> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation block hook 0x55801f1afc80 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -80> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation histogram block hook 0x55801f1afc80 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -79> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator dump recovery hook 0x55801f444030 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -78> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator score recovery hook 0x55801f444030 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -77> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation recovery hook 0x55801f444030 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -76> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation histogram recovery hook 0x55801f444030 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -75> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::__restore_allocator::file_size=132,sizeof(extent_t)=16 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -74> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::__restore_allocator::READ--extent_count=1, read_alloc_size= 107374174208, file_size=132 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -73> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::__restore_allocator::READ duration=0.000461 seconds, s_serial=3 2026-02-17T21:11:37.241 INFO:tasks.workunit.client.0.trial060.stderr: -72> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::restore_allocator:: calling copy_allocator(bitmap_allocator -> shared_alloc.a) 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -71> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::copy_allocator::count num_entries=1 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -70> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::copy_allocator::copy num_entries=1 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -69> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::restore_allocator::restored in 0.000716 seconds, num_entries=1 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -68> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator dump recovery 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -67> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator fragmentation histogram recovery 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -66> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator fragmentation recovery 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -65> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator score recovery 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -64> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() completed successfully alloc=0x55801f3cb200 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -63> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -62> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bluefs umount 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -61> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluefs debug_inject_read_zeros 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -60> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluefs files list 2026-02-17T21:11:37.242 INFO:tasks.workunit.client.0.trial060.stderr: -59> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluefs stats 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -58> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore bluefs device info 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -57> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) close 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -56> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore bluefs device info hook 0x55801f024cd0 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -55> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs stats hook 0x55801f024cd0 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -54> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs files list hook 0x55801f024cd0 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -53> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs debug_inject_read_zeros hook 0x55801f024cd0 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -52> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -51> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 0 bdev(0x55801f1b0000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -50> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -49> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -48> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs mount 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -47> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 bluefs mount super: super(uuid fcbec683-5f94-4994-8bd5-75eefc73e648 osd 3d806266-f149-4138-830d-01af17cfcc64 seq 4 block_size 0x1000 log_fnode 0xfile(ino 1 size 0x0 mtime 2026-02-17T21:11:36.170492+0000 allocated 10000 alloc_commit 10000 extents [1:0xc2000~10000])) 2026-02-17T21:11:37.243 INFO:tasks.workunit.client.0.trial060.stderr: -46> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -45> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -44> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -43> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -42> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -41> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -40> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount shared_bdev_used = 4915200 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -39> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -38> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -37> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -36> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -35> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -34> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.244 INFO:tasks.workunit.client.0.trial060.stderr: -33> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -32> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -31> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -30> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -29> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -28> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -27> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -26> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 5 bluestore::NCB::invalidate_allocation_file_on_bluefs::invalidate using bluefs->truncate(p_handle, 0) 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -25> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -24> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -23> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -22> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -21> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -20> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.245 INFO:tasks.workunit.client.0.trial060.stderr: -19> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -18> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -17> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -16> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -15> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -14> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -13> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -12> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -11> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -10> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -9> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -8> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.246 INFO:tasks.workunit.client.0.trial060.stderr: -7> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: -6> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: -5> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: -4> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: -3> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: -2> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: -1> 2026-02-17T21:11:37.217+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) umount 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: 0> 2026-02-17T21:11:37.217+0000 7fb8747d0a80 -1 *** Caught signal (Segmentation fault) ** 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: in thread 7fb8747d0a80 thread_name:ceph-kvstore-to 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable) 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: 1: /lib64/libc.so.6(+0x3fc30) [0x7fb87483fc30] 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xb4) [0x55801d26ce54] 2026-02-17T21:11:37.247 INFO:tasks.workunit.client.0.trial060.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string, std::allocator > const&, unsigned int, KeyValueDB::IteratorBounds)+0x240) [0x55801d26d570] 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 4: (BlueStore::_close_db()+0x380) [0x55801d0aa470] 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 5: (BlueStore::umount()+0x244) [0x55801d0b6fe4] 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 6: ceph-kvstore-tool(+0x1f7503) [0x55801d054503] 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 7: main() 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 8: /lib64/libc.so.6(+0x2a610) [0x7fb87482a610] 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 9: __libc_start_main() 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 10: _start() 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr:--- logging levels --- 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 none 2026-02-17T21:11:37.248 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 lockdep 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 context 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 crush 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_balancer 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_locker 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_log 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_log_expire 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_migrator 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 3/ 5 mds_quiesce 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 buffer 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 timer 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 filer 2026-02-17T21:11:37.249 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 striper 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 objecter 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rados 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd_mirror 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd_replay 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd_pwl 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 journaler 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 objectcacher 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 immutable_obj_cache 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 client 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 osd 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 optracker 2026-02-17T21:11:37.250 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 objclass 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 3 filestore 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 3 journal 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 0 ms 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mon 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 0/10 monc 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 paxos 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 tp 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 auth 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 crypto 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 finisher 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 reserver 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 heartbeatmap 2026-02-17T21:11:37.251 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 perfcounter 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_sync 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_datacache 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_access 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_dbstore 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_flight 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_lifecycle 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_notification 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 javaclient 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 asok 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 throttle 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 0 refs 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 compressor 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 bluestore 2026-02-17T21:11:37.252 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 bluefs 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 3 bdev 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 kstore 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 2/ 2 rocksdb 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 fuse 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 2/ 5 mgr 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mgrc 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 dpdk 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 eventtrace 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 prioritycache 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 test 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 cephfs_mirror 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 cephsqlite 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 crimson_interrupt 2026-02-17T21:11:37.253 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_onode 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_odata 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_omap 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_tm 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_t 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_cleaner 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_epm 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_lba 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_cache 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_journal 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_device 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_backref 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 alienstore 2026-02-17T21:11:37.254 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mclock 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 cyanstore 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 ceph_exporter 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 memstore 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 trace 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: -2/-2 (syslog threshold) 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: 99/99 (stderr threshold) 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: 7fb8747d0a80 / ceph-kvstore-to 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: max_recent 500 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: max_new 1000 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: log_file 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr:--- end dump of recent events --- 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr:--- begin dump of recent events --- 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: -194> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command assert hook 0x55801f0249c0 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: -193> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command abort hook 0x55801f0249c0 2026-02-17T21:11:37.255 INFO:tasks.workunit.client.0.trial060.stderr: -192> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command leak_some_memory hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -191> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perfcounters_dump hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -190> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command 1 hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -189> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf dump hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -188> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perfcounters_schema hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -187> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf histogram dump hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -186> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command 2 hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -185> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf schema hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -184> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command counter dump hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -183> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command counter schema hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -182> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf histogram schema hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -181> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command perf reset hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -180> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config show hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -179> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config help hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -178> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config set hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -177> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config unset hook 0x55801f0249c0 2026-02-17T21:11:37.256 INFO:tasks.workunit.client.0.trial060.stderr: -176> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config get hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -175> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config diff hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -174> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command config diff get hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -173> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command injectargs hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -172> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command log flush hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -171> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command log dump hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -170> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command log reopen hook 0x55801f0249c0 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -169> 2026-02-17T21:11:36.665+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command dump_mempools hook 0x55801f104068 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -168> 2026-02-17T21:11:36.667+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _open_db_and_around::NCB::read_only=0, to_repair=1 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -167> 2026-02-17T21:11:36.667+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -166> 2026-02-17T21:11:36.667+0000 7fb8747d0a80 0 bdev(0x55801f031c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -165> 2026-02-17T21:11:36.668+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -164> 2026-02-17T21:11:36.668+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) close 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -163> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -162> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 0 bdev(0x55801f031c00 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -161> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f031c00 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.257 INFO:tasks.workunit.client.0.trial060.stderr: -160> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -159> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore bluefs device info hook 0x55801f0249e0 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -158> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs stats hook 0x55801f0249e0 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -157> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs files list hook 0x55801f0249e0 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -156> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs debug_inject_read_zeros hook 0x55801f0249e0 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -155> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -154> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 0 bdev(0x55801f1b0000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -153> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -152> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -151> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs mount 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -150> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 5 bluefs mount super: super(uuid fcbec683-5f94-4994-8bd5-75eefc73e648 osd 3d806266-f149-4138-830d-01af17cfcc64 seq 4 block_size 0x1000 log_fnode 0xfile(ino 1 size 0x0 mtime 2026-02-17T21:11:36.170492+0000 allocated 10000 alloc_commit 10000 extents [1:0xc2000~10000])) 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -149> 2026-02-17T21:11:36.930+0000 7fb8747d0a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -148> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.258 INFO:tasks.workunit.client.0.trial060.stderr: -147> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -146> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -145> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -144> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -143> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluefs mount shared_bdev_used = 0 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -142> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -141> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -140> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -139> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -138> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -137> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -136> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -135> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -134> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -133> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -132> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -131> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.259 INFO:tasks.workunit.client.0.trial060.stderr: -130> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -129> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -128> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -127> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -126> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -125> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -124> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -123> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -122> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -121> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -120> 2026-02-17T21:11:36.933+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -119> 2026-02-17T21:11:36.934+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -118> 2026-02-17T21:11:36.934+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -117> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -116> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta old nid_max 0 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -115> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta old blobid_max 0 2026-02-17T21:11:37.260 INFO:tasks.workunit.client.0.trial060.stderr: -114> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _open_super_meta::NCB::freelist_type=null 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -113> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -112> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _open_super_meta min_alloc_size 0x1000 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -111> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -110> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor snappy 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -109> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor snappy 0x55801f1af530 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -108> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0x55801f1af530 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -107> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor snappy loaded and registered 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -106> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0x55801f1af530 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -105> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zlib = 0 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -104> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor zlib 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -103> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor zlib 0x55801f1af6e0 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -102> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zlib = 0x55801f1af6e0 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -101> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor zlib loaded and registered 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -100> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zlib = 0x55801f1af6e0 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -99> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zstd = 0 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -98> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor zstd 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -97> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor zstd 0x55801f1af770 2026-02-17T21:11:37.261 INFO:tasks.workunit.client.0.trial060.stderr: -96> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zstd = 0x55801f1af770 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -95> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor zstd loaded and registered 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -94> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor zstd = 0x55801f1af770 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -93> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor lz4 = 0 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -92> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load compressor lz4 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -91> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 add compressor lz4 0x55801f1afcb0 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -90> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor lz4 = 0x55801f1afcb0 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -89> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 load: compressor lz4 loaded and registered 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -88> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor lz4 = 0x55801f1afcb0 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -87> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 get compressor snappy = 0x55801f1af530 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -86> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _open_fm::NCB::freelist_type=null 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -85> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 freelist init 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -84> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 1 freelist _read_cfg 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -83> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator dump block hook 0x55801f1afc80 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -82> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator score block hook 0x55801f1afc80 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -81> 2026-02-17T21:11:36.939+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation block hook 0x55801f1afc80 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -80> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation histogram block hook 0x55801f1afc80 2026-02-17T21:11:37.262 INFO:tasks.workunit.client.0.trial060.stderr: -79> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator dump recovery hook 0x55801f444030 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -78> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator score recovery hook 0x55801f444030 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -77> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation recovery hook 0x55801f444030 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -76> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore allocator fragmentation histogram recovery hook 0x55801f444030 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -75> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::__restore_allocator::file_size=132,sizeof(extent_t)=16 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -74> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::__restore_allocator::READ--extent_count=1, read_alloc_size= 107374174208, file_size=132 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -73> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::__restore_allocator::READ duration=0.000461 seconds, s_serial=3 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -72> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::restore_allocator:: calling copy_allocator(bitmap_allocator -> shared_alloc.a) 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -71> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::copy_allocator::count num_entries=1 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -70> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::copy_allocator::copy num_entries=1 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -69> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore::NCB::restore_allocator::restored in 0.000716 seconds, num_entries=1 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -68> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator dump recovery 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -67> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator fragmentation histogram recovery 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -66> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator fragmentation recovery 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -65> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore allocator score recovery 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -64> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) _init_alloc::NCB::restore_allocator() completed successfully alloc=0x55801f3cb200 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -63> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _init_alloc loaded 100 GiB in 1 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -62> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bluefs umount 2026-02-17T21:11:37.263 INFO:tasks.workunit.client.0.trial060.stderr: -61> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluefs debug_inject_read_zeros 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -60> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluefs files list 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -59> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluefs stats 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -58> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 5 asok(0x55801f0bc000) unregister_commands bluestore bluefs device info 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -57> 2026-02-17T21:11:36.940+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) close 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -56> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluestore bluefs device info hook 0x55801f024cd0 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -55> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs stats hook 0x55801f024cd0 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -54> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs files list hook 0x55801f024cd0 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -53> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 asok(0x55801f0bc000) register_command bluefs debug_inject_read_zeros hook 0x55801f024cd0 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -52> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open path ./cephtool.NM8/block 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -51> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 0 bdev(0x55801f1b0000 ./cephtool.NM8/block) ioctl(F_SET_FILE_RW_HINT) on ./cephtool.NM8/block failed: (22) Invalid argument 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -50> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bdev(0x55801f1b0000 ./cephtool.NM8/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -49> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs add_block_device bdev 1 path ./cephtool.NM8/block size 100 GiB 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -48> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs mount 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -47> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 5 bluefs mount super: super(uuid fcbec683-5f94-4994-8bd5-75eefc73e648 osd 3d806266-f149-4138-830d-01af17cfcc64 seq 4 block_size 0x1000 log_fnode 0xfile(ino 1 size 0x0 mtime 2026-02-17T21:11:36.170492+0000 allocated 10000 alloc_commit 10000 extents [1:0xc2000~10000])) 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -46> 2026-02-17T21:11:37.196+0000 7fb8747d0a80 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -45> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -44> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.264 INFO:tasks.workunit.client.0.trial060.stderr: -43> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -42> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -41> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -40> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluefs mount shared_bdev_used = 4915200 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -39> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 bluestore(./cephtool.NM8) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -38> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -37> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -36> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -35> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -34> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -33> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -32> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -31> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -30> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -29> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -28> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -27> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -26> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 5 bluestore::NCB::invalidate_allocation_file_on_bluefs::invalidate using bluefs->truncate(p_handle, 0) 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -25> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.265 INFO:tasks.workunit.client.0.trial060.stderr: -24> 2026-02-17T21:11:37.197+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -23> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -22> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -21> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -20> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -19> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -18> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -17> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -16> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -15> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -14> 2026-02-17T21:11:37.198+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -13> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option writable_file_max_buffer_size = 0 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -12> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_base = 1073741824 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -11> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option level0_file_num_compaction_trigger = 8 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -10> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_background_jobs = 4 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -9> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option write_buffer_size = 16777216 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -8> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option compaction_style = kCompactionStyleLevel 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -7> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option compaction_readahead_size = 2MB 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -6> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_bytes_for_level_multiplier = 8 2026-02-17T21:11:37.266 INFO:tasks.workunit.client.0.trial060.stderr: -5> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_write_buffer_number = 64 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: -4> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option max_total_wal_size = 1073741824 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: -3> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option min_write_buffer_number_to_merge = 6 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: -2> 2026-02-17T21:11:37.200+0000 7fb8747d0a80 1 set rocksdb option compression = kLZ4Compression 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: -1> 2026-02-17T21:11:37.217+0000 7fb8747d0a80 5 bluestore(./cephtool.NM8) umount 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 0> 2026-02-17T21:11:37.217+0000 7fb8747d0a80 -1 *** Caught signal (Segmentation fault) ** 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: in thread 7fb8747d0a80 thread_name:ceph-kvstore-to 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable) 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 1: /lib64/libc.so.6(+0x3fc30) [0x7fb87483fc30] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xb4) [0x55801d26ce54] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string, std::allocator > const&, unsigned int, KeyValueDB::IteratorBounds)+0x240) [0x55801d26d570] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 4: (BlueStore::_close_db()+0x380) [0x55801d0aa470] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 5: (BlueStore::umount()+0x244) [0x55801d0b6fe4] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 6: ceph-kvstore-tool(+0x1f7503) [0x55801d054503] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 7: main() 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 8: /lib64/libc.so.6(+0x2a610) [0x7fb87482a610] 2026-02-17T21:11:37.267 INFO:tasks.workunit.client.0.trial060.stderr: 9: __libc_start_main() 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 10: _start() 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr:--- logging levels --- 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 none 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 lockdep 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 context 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 crush 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_balancer 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_locker 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_log 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_log_expire 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mds_migrator 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 3/ 5 mds_quiesce 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 buffer 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 timer 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 filer 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 striper 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 1 objecter 2026-02-17T21:11:37.268 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rados 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd_mirror 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd_replay 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 rbd_pwl 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 journaler 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 objectcacher 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 immutable_obj_cache 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 client 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 osd 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 optracker 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 objclass 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 3 filestore 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 3 journal 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 0 ms 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mon 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/10 monc 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 paxos 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 tp 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 auth 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 crypto 2026-02-17T21:11:37.269 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 finisher 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 reserver 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 heartbeatmap 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 perfcounter 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_sync 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_datacache 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_access 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_dbstore 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_flight 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_lifecycle 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 rgw_notification 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 javaclient 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 asok 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 1 throttle 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 0 refs 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 compressor 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 bluestore 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 bluefs 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 3 bdev 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 kstore 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 2/ 2 rocksdb 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 fuse 2026-02-17T21:11:37.270 INFO:tasks.workunit.client.0.trial060.stderr: 2/ 5 mgr 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mgrc 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 dpdk 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 eventtrace 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 prioritycache 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 test 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 cephfs_mirror 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 cephsqlite 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 crimson_interrupt 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_onode 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_odata 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_omap 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_tm 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_t 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_cleaner 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_epm 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_lba 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_cache 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_journal 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_device 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 seastore_backref 2026-02-17T21:11:37.271 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 alienstore 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 mclock 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 0/ 5 cyanstore 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 ceph_exporter 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 memstore 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 1/ 5 trace 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: -2/-2 (syslog threshold) 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 99/99 (stderr threshold) 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: 7fb8747d0a80 / ceph-kvstore-to 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: max_recent 500 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: max_new 1000 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr: log_file /var/lib/ceph/crash/2026-02-17T21:11:37.218796Z_18267bac-7982-4490-aba2-9dd66fa1ffaf/log 2026-02-17T21:11:37.272 INFO:tasks.workunit.client.0.trial060.stderr:--- end dump of recent events --- 2026-02-17T21:11:37.273 DEBUG:teuthology.orchestra.run:got remote process result: 139 2026-02-17T21:11:37.273 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh: line 27: 31925 Segmentation fault (core dumped) ceph-kvstore-tool bluestore-kv ${TEMP_DIR} destructive-repair 2026-02-17T21:11:37.273 INFO:tasks.workunit.client.0.trial060.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:1: test_ceph_kvstore_tool: rm -fr ./cephtool.NM8 2026-02-17T21:11:37.273 INFO:tasks.workunit:Stopping ['cephtool/test_kvstore_tool.sh'] on client.0... 2026-02-17T21:11:37.273 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T21:11:37.588 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test cephtool/test_kvstore_tool.sh) on trial060 with status 139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh' 2026-02-17T21:11:37.592 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-17T21:11:37.602 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T21:11:37.602 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:11:37.760 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:37.761 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:11:37.769 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-17T21:11:36.838210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81260,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491540,"statfs":{"total":322122547200,"available":322039336960,"internally_reserved":0,"allocated":376832,"data_stored":98126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001753"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585382+0000","last_change":"2026-02-17T21:10:58.048707+0000","last_active":"2026-02-17T21:11:00.585382+0000","last_peered":"2026-02-17T21:11:00.585382+0000","last_clean":"2026-02-17T21:11:00.585382+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:11:00.585382+0000","last_undegraded":"2026-02-17T21:11:00.585382+0000","last_fullsized":"2026-02-17T21:11:00.585382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:56:39.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585374+0000","last_change":"2026-02-17T21:10:58.048745+0000","last_active":"2026-02-17T21:11:00.585374+0000","last_peered":"2026-02-17T21:11:00.585374+0000","last_clean":"2026-02-17T21:11:00.585374+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:11:00.585374+0000","last_undegraded":"2026-02-17T21:11:00.585374+0000","last_fullsized":"2026-02-17T21:11:00.585374+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T06:49:23.558997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142995+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142995+0000","last_peered":"2026-02-17T21:11:01.142995+0000","last_clean":"2026-02-17T21:11:01.142995+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:11:01.142995+0000","last_undegraded":"2026-02-17T21:11:01.142995+0000","last_fullsized":"2026-02-17T21:11:01.142995+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:35:26.275365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585381+0000","last_change":"2026-02-17T21:10:58.048849+0000","last_active":"2026-02-17T21:11:00.585381+0000","last_peered":"2026-02-17T21:11:00.585381+0000","last_clean":"2026-02-17T21:11:00.585381+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:11:00.585381+0000","last_undegraded":"2026-02-17T21:11:00.585381+0000","last_fullsized":"2026-02-17T21:11:00.585381+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:21:32.893769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585379+0000","last_change":"2026-02-17T21:10:58.048464+0000","last_active":"2026-02-17T21:11:00.585379+0000","last_peered":"2026-02-17T21:11:00.585379+0000","last_clean":"2026-02-17T21:11:00.585379+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:11:00.585379+0000","last_undegraded":"2026-02-17T21:11:00.585379+0000","last_fullsized":"2026-02-17T21:11:00.585379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T08:31:58.499464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.734240+0000","last_change":"2026-02-17T21:10:58.049445+0000","last_active":"2026-02-17T21:11:00.734240+0000","last_peered":"2026-02-17T21:11:00.734240+0000","last_clean":"2026-02-17T21:11:00.734240+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:11:00.734240+0000","last_undegraded":"2026-02-17T21:11:00.734240+0000","last_fullsized":"2026-02-17T21:11:00.734240+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T07:53:55.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142977+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142977+0000","last_peered":"2026-02-17T21:11:01.142977+0000","last_clean":"2026-02-17T21:11:01.142977+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:11:01.142977+0000","last_undegraded":"2026-02-17T21:11:01.142977+0000","last_fullsized":"2026-02-17T21:11:01.142977+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T02:30:55.687423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585355+0000","last_change":"2026-02-17T21:10:58.048469+0000","last_active":"2026-02-17T21:11:00.585355+0000","last_peered":"2026-02-17T21:11:00.585355+0000","last_clean":"2026-02-17T21:11:00.585355+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:11:00.585355+0000","last_undegraded":"2026-02-17T21:11:00.585355+0000","last_fullsized":"2026-02-17T21:11:00.585355+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:08:41.667195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705677,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738380,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27084,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830516,"statfs":{"total":107374182400,"available":107346448384,"internally_reserved":0,"allocated":122880,"data_stored":32696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738380,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:11:37.770 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:11:37.928 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:37.928 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:11:37.937 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-17T21:11:36.838210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81260,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491540,"statfs":{"total":322122547200,"available":322039336960,"internally_reserved":0,"allocated":376832,"data_stored":98126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001753"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585382+0000","last_change":"2026-02-17T21:10:58.048707+0000","last_active":"2026-02-17T21:11:00.585382+0000","last_peered":"2026-02-17T21:11:00.585382+0000","last_clean":"2026-02-17T21:11:00.585382+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:11:00.585382+0000","last_undegraded":"2026-02-17T21:11:00.585382+0000","last_fullsized":"2026-02-17T21:11:00.585382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:56:39.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585374+0000","last_change":"2026-02-17T21:10:58.048745+0000","last_active":"2026-02-17T21:11:00.585374+0000","last_peered":"2026-02-17T21:11:00.585374+0000","last_clean":"2026-02-17T21:11:00.585374+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:11:00.585374+0000","last_undegraded":"2026-02-17T21:11:00.585374+0000","last_fullsized":"2026-02-17T21:11:00.585374+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T06:49:23.558997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142995+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142995+0000","last_peered":"2026-02-17T21:11:01.142995+0000","last_clean":"2026-02-17T21:11:01.142995+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:11:01.142995+0000","last_undegraded":"2026-02-17T21:11:01.142995+0000","last_fullsized":"2026-02-17T21:11:01.142995+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:35:26.275365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585381+0000","last_change":"2026-02-17T21:10:58.048849+0000","last_active":"2026-02-17T21:11:00.585381+0000","last_peered":"2026-02-17T21:11:00.585381+0000","last_clean":"2026-02-17T21:11:00.585381+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:11:00.585381+0000","last_undegraded":"2026-02-17T21:11:00.585381+0000","last_fullsized":"2026-02-17T21:11:00.585381+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:21:32.893769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585379+0000","last_change":"2026-02-17T21:10:58.048464+0000","last_active":"2026-02-17T21:11:00.585379+0000","last_peered":"2026-02-17T21:11:00.585379+0000","last_clean":"2026-02-17T21:11:00.585379+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:11:00.585379+0000","last_undegraded":"2026-02-17T21:11:00.585379+0000","last_fullsized":"2026-02-17T21:11:00.585379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T08:31:58.499464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.734240+0000","last_change":"2026-02-17T21:10:58.049445+0000","last_active":"2026-02-17T21:11:00.734240+0000","last_peered":"2026-02-17T21:11:00.734240+0000","last_clean":"2026-02-17T21:11:00.734240+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:11:00.734240+0000","last_undegraded":"2026-02-17T21:11:00.734240+0000","last_fullsized":"2026-02-17T21:11:00.734240+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T07:53:55.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142977+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142977+0000","last_peered":"2026-02-17T21:11:01.142977+0000","last_clean":"2026-02-17T21:11:01.142977+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:11:01.142977+0000","last_undegraded":"2026-02-17T21:11:01.142977+0000","last_fullsized":"2026-02-17T21:11:01.142977+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T02:30:55.687423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585355+0000","last_change":"2026-02-17T21:10:58.048469+0000","last_active":"2026-02-17T21:11:00.585355+0000","last_peered":"2026-02-17T21:11:00.585355+0000","last_clean":"2026-02-17T21:11:00.585355+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:11:00.585355+0000","last_undegraded":"2026-02-17T21:11:00.585355+0000","last_fullsized":"2026-02-17T21:11:00.585355+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:08:41.667195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705677,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738380,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27084,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830516,"statfs":{"total":107374182400,"available":107346448384,"internally_reserved":0,"allocated":122880,"data_stored":32696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738380,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:11:37.938 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T21:11:37.938 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:11:38.096 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:38.097 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:11:38.105 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-17T21:11:36.838210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81260,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491540,"statfs":{"total":322122547200,"available":322039336960,"internally_reserved":0,"allocated":376832,"data_stored":98126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001753"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585382+0000","last_change":"2026-02-17T21:10:58.048707+0000","last_active":"2026-02-17T21:11:00.585382+0000","last_peered":"2026-02-17T21:11:00.585382+0000","last_clean":"2026-02-17T21:11:00.585382+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:11:00.585382+0000","last_undegraded":"2026-02-17T21:11:00.585382+0000","last_fullsized":"2026-02-17T21:11:00.585382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:56:39.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585374+0000","last_change":"2026-02-17T21:10:58.048745+0000","last_active":"2026-02-17T21:11:00.585374+0000","last_peered":"2026-02-17T21:11:00.585374+0000","last_clean":"2026-02-17T21:11:00.585374+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:11:00.585374+0000","last_undegraded":"2026-02-17T21:11:00.585374+0000","last_fullsized":"2026-02-17T21:11:00.585374+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T06:49:23.558997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142995+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142995+0000","last_peered":"2026-02-17T21:11:01.142995+0000","last_clean":"2026-02-17T21:11:01.142995+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:11:01.142995+0000","last_undegraded":"2026-02-17T21:11:01.142995+0000","last_fullsized":"2026-02-17T21:11:01.142995+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:35:26.275365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585381+0000","last_change":"2026-02-17T21:10:58.048849+0000","last_active":"2026-02-17T21:11:00.585381+0000","last_peered":"2026-02-17T21:11:00.585381+0000","last_clean":"2026-02-17T21:11:00.585381+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:11:00.585381+0000","last_undegraded":"2026-02-17T21:11:00.585381+0000","last_fullsized":"2026-02-17T21:11:00.585381+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:21:32.893769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585379+0000","last_change":"2026-02-17T21:10:58.048464+0000","last_active":"2026-02-17T21:11:00.585379+0000","last_peered":"2026-02-17T21:11:00.585379+0000","last_clean":"2026-02-17T21:11:00.585379+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:11:00.585379+0000","last_undegraded":"2026-02-17T21:11:00.585379+0000","last_fullsized":"2026-02-17T21:11:00.585379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T08:31:58.499464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.734240+0000","last_change":"2026-02-17T21:10:58.049445+0000","last_active":"2026-02-17T21:11:00.734240+0000","last_peered":"2026-02-17T21:11:00.734240+0000","last_clean":"2026-02-17T21:11:00.734240+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:11:00.734240+0000","last_undegraded":"2026-02-17T21:11:00.734240+0000","last_fullsized":"2026-02-17T21:11:00.734240+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T07:53:55.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142977+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142977+0000","last_peered":"2026-02-17T21:11:01.142977+0000","last_clean":"2026-02-17T21:11:01.142977+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:11:01.142977+0000","last_undegraded":"2026-02-17T21:11:01.142977+0000","last_fullsized":"2026-02-17T21:11:01.142977+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T02:30:55.687423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585355+0000","last_change":"2026-02-17T21:10:58.048469+0000","last_active":"2026-02-17T21:11:00.585355+0000","last_peered":"2026-02-17T21:11:00.585355+0000","last_clean":"2026-02-17T21:11:00.585355+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:11:00.585355+0000","last_undegraded":"2026-02-17T21:11:00.585355+0000","last_fullsized":"2026-02-17T21:11:00.585355+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:08:41.667195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705677,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738380,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27084,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830516,"statfs":{"total":107374182400,"available":107346448384,"internally_reserved":0,"allocated":122880,"data_stored":32696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738380,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:11:38.106 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T21:11:38.264 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:38.264 INFO:teuthology.orchestra.run.trial060.stdout:{"epoch":14,"fsid":"693a7425-f499-41f0-a5e5-0a2aa082bd73","created":"2026-02-17T21:10:40.588757+0000","modified":"2026-02-17T21:11:00.583807+0000","last_up_change":"2026-02-17T21:10:49.035977+0000","last_in_change":"2026-02-17T21:10:41.264747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-17T21:10:54.874321+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1314774c-e40f-425a-9715-3d5104938d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6810","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6811","nonce":1320383713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6812","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6813","nonce":1320383713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6816","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6817","nonce":1320383713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6814","nonce":1320383713},{"type":"v1","addr":"10.20.193.60:6815","nonce":1320383713}]},"public_addr":"10.20.193.60:6811/1320383713","cluster_addr":"10.20.193.60:6813/1320383713","heartbeat_back_addr":"10.20.193.60:6817/1320383713","heartbeat_front_addr":"10.20.193.60:6815/1320383713","state":["exists","up"]},{"osd":1,"uuid":"416b341c-8e2f-4371-a402-e379d3799a48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6818","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6819","nonce":2878742521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6820","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6821","nonce":2878742521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6824","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6825","nonce":2878742521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6822","nonce":2878742521},{"type":"v1","addr":"10.20.193.60:6823","nonce":2878742521}]},"public_addr":"10.20.193.60:6819/2878742521","cluster_addr":"10.20.193.60:6821/2878742521","heartbeat_back_addr":"10.20.193.60:6825/2878742521","heartbeat_front_addr":"10.20.193.60:6823/2878742521","state":["exists","up"]},{"osd":2,"uuid":"d0157467-2bcb-4a82-aa65-a9f16d18a519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6800","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6801","nonce":633340544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6802","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6803","nonce":633340544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6806","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6807","nonce":633340544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.60:6804","nonce":633340544},{"type":"v1","addr":"10.20.193.60:6805","nonce":633340544}]},"public_addr":"10.20.193.60:6801/633340544","cluster_addr":"10.20.193.60:6803/633340544","heartbeat_back_addr":"10.20.193.60:6807/633340544","heartbeat_front_addr":"10.20.193.60:6805/633340544","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T21:11:39.275 INFO:tasks.ceph:Scrubbing osd.0 2026-02-17T21:11:39.275 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-17T21:11:39.356 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-17T21:11:39.356 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T21:11:39.356 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-17T21:11:39.361 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-17T21:11:39.521 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 0 to deep-scrub 2026-02-17T21:11:39.530 INFO:tasks.ceph:Scrubbing osd.1 2026-02-17T21:11:39.530 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-17T21:11:39.611 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-17T21:11:39.611 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T21:11:39.611 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-17T21:11:39.617 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-17T21:11:39.778 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 1 to deep-scrub 2026-02-17T21:11:39.786 INFO:tasks.ceph:Scrubbing osd.2 2026-02-17T21:11:39.786 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-17T21:11:39.865 INFO:teuthology.orchestra.run.trial060.stdout:{ 2026-02-17T21:11:39.865 INFO:teuthology.orchestra.run.trial060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-17T21:11:39.865 INFO:teuthology.orchestra.run.trial060.stdout:} 2026-02-17T21:11:39.870 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-17T21:11:40.026 INFO:teuthology.orchestra.run.trial060.stderr:instructed osd(s) 2 to deep-scrub 2026-02-17T21:11:40.035 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:11:40.192 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:11:40.193 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:11:40.200 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2026-02-17T21:11:38.838533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81260,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491540,"statfs":{"total":322122547200,"available":322039336960,"internally_reserved":0,"allocated":376832,"data_stored":98126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001805"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585382+0000","last_change":"2026-02-17T21:10:58.048707+0000","last_active":"2026-02-17T21:11:00.585382+0000","last_peered":"2026-02-17T21:11:00.585382+0000","last_clean":"2026-02-17T21:11:00.585382+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:11:00.585382+0000","last_undegraded":"2026-02-17T21:11:00.585382+0000","last_fullsized":"2026-02-17T21:11:00.585382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:56:39.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585374+0000","last_change":"2026-02-17T21:10:58.048745+0000","last_active":"2026-02-17T21:11:00.585374+0000","last_peered":"2026-02-17T21:11:00.585374+0000","last_clean":"2026-02-17T21:11:00.585374+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:11:00.585374+0000","last_undegraded":"2026-02-17T21:11:00.585374+0000","last_fullsized":"2026-02-17T21:11:00.585374+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T06:49:23.558997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142995+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142995+0000","last_peered":"2026-02-17T21:11:01.142995+0000","last_clean":"2026-02-17T21:11:01.142995+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:11:01.142995+0000","last_undegraded":"2026-02-17T21:11:01.142995+0000","last_fullsized":"2026-02-17T21:11:01.142995+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:35:26.275365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585381+0000","last_change":"2026-02-17T21:10:58.048849+0000","last_active":"2026-02-17T21:11:00.585381+0000","last_peered":"2026-02-17T21:11:00.585381+0000","last_clean":"2026-02-17T21:11:00.585381+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:11:00.585381+0000","last_undegraded":"2026-02-17T21:11:00.585381+0000","last_fullsized":"2026-02-17T21:11:00.585381+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:21:32.893769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585379+0000","last_change":"2026-02-17T21:10:58.048464+0000","last_active":"2026-02-17T21:11:00.585379+0000","last_peered":"2026-02-17T21:11:00.585379+0000","last_clean":"2026-02-17T21:11:00.585379+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:11:00.585379+0000","last_undegraded":"2026-02-17T21:11:00.585379+0000","last_fullsized":"2026-02-17T21:11:00.585379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T08:31:58.499464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.734240+0000","last_change":"2026-02-17T21:10:58.049445+0000","last_active":"2026-02-17T21:11:00.734240+0000","last_peered":"2026-02-17T21:11:00.734240+0000","last_clean":"2026-02-17T21:11:00.734240+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:11:00.734240+0000","last_undegraded":"2026-02-17T21:11:00.734240+0000","last_fullsized":"2026-02-17T21:11:00.734240+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T07:53:55.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:01.142977+0000","last_change":"2026-02-17T21:10:58.403864+0000","last_active":"2026-02-17T21:11:01.142977+0000","last_peered":"2026-02-17T21:11:01.142977+0000","last_clean":"2026-02-17T21:11:01.142977+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:11:01.142977+0000","last_undegraded":"2026-02-17T21:11:01.142977+0000","last_fullsized":"2026-02-17T21:11:01.142977+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T02:30:55.687423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:00.585355+0000","last_change":"2026-02-17T21:10:58.048469+0000","last_active":"2026-02-17T21:11:00.585355+0000","last_peered":"2026-02-17T21:11:00.585355+0000","last_clean":"2026-02-17T21:11:00.585355+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:11:00.585355+0000","last_undegraded":"2026-02-17T21:11:00.585355+0000","last_fullsized":"2026-02-17T21:11:00.585355+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:10:55.038954+0000","last_clean_scrub_stamp":"2026-02-17T21:10:55.038954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:08:41.667195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705677,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738381,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27084,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830516,"statfs":{"total":107374182400,"available":107346448384,"internally_reserved":0,"allocated":122880,"data_stored":32696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738381,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:11:40.205 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.205 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-17T21:10:55.038954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=10, tm_sec=55, tm_wday=1, tm_yday=48, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=17, tm_hour=21, tm_min=11, tm_sec=38, tm_wday=1, tm_yday=48, tm_isdst=0) 2026-02-17T21:11:40.206 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-17T21:12:00.208 DEBUG:teuthology.orchestra.run.trial060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T21:12:00.365 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-17T21:12:00.366 INFO:teuthology.orchestra.run.trial060.stderr:dumped all 2026-02-17T21:12:00.374 INFO:teuthology.orchestra.run.trial060.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2026-02-17T21:11:58.840316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":81248,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314491552,"statfs":{"total":322122547200,"available":322039349248,"internally_reserved":0,"allocated":376832,"data_stored":98126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000907"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:40.274903+0000","last_change":"2026-02-17T21:11:40.274917+0000","last_active":"2026-02-17T21:11:40.274903+0000","last_peered":"2026-02-17T21:11:40.274903+0000","last_clean":"2026-02-17T21:11:40.274903+0000","last_became_active":"2026-02-17T21:10:56.047479+0000","last_became_peered":"2026-02-17T21:10:56.047479+0000","last_unstale":"2026-02-17T21:11:40.274903+0000","last_undegraded":"2026-02-17T21:11:40.274903+0000","last_fullsized":"2026-02-17T21:11:40.274903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:11:40.274837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:11:40.274837+0000","last_clean_scrub_stamp":"2026-02-17T21:11:40.274837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T05:27:50.160611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056662499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":30,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:41.314167+0000","last_change":"2026-02-17T21:11:41.314177+0000","last_active":"2026-02-17T21:11:41.314167+0000","last_peered":"2026-02-17T21:11:41.314167+0000","last_clean":"2026-02-17T21:11:41.314167+0000","last_became_active":"2026-02-17T21:10:56.047831+0000","last_became_peered":"2026-02-17T21:10:56.047831+0000","last_unstale":"2026-02-17T21:11:41.314167+0000","last_undegraded":"2026-02-17T21:11:41.314167+0000","last_fullsized":"2026-02-17T21:11:41.314167+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:11:41.314115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:11:41.314115+0000","last_clean_scrub_stamp":"2026-02-17T21:11:41.314115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T23:06:28.160670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058379799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:41.689175+0000","last_change":"2026-02-17T21:11:41.689194+0000","last_active":"2026-02-17T21:11:41.689175+0000","last_peered":"2026-02-17T21:11:41.689175+0000","last_clean":"2026-02-17T21:11:41.689175+0000","last_became_active":"2026-02-17T21:10:56.047863+0000","last_became_peered":"2026-02-17T21:10:56.047863+0000","last_unstale":"2026-02-17T21:11:41.689175+0000","last_undegraded":"2026-02-17T21:11:41.689175+0000","last_fullsized":"2026-02-17T21:11:41.689175+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:11:41.689092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:11:41.689092+0000","last_clean_scrub_stamp":"2026-02-17T21:11:41.689092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T04:54:18.676932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000214219,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:44.331062+0000","last_change":"2026-02-17T21:11:44.331076+0000","last_active":"2026-02-17T21:11:44.331062+0000","last_peered":"2026-02-17T21:11:44.331062+0000","last_clean":"2026-02-17T21:11:44.331062+0000","last_became_active":"2026-02-17T21:10:56.048020+0000","last_became_peered":"2026-02-17T21:10:56.048020+0000","last_unstale":"2026-02-17T21:11:44.331062+0000","last_undegraded":"2026-02-17T21:11:44.331062+0000","last_fullsized":"2026-02-17T21:11:44.331062+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:11:44.331016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:11:44.331016+0000","last_clean_scrub_stamp":"2026-02-17T21:11:44.331016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T22:45:44.240782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063101599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":32,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:42.320442+0000","last_change":"2026-02-17T21:11:42.320454+0000","last_active":"2026-02-17T21:11:42.320442+0000","last_peered":"2026-02-17T21:11:42.320442+0000","last_clean":"2026-02-17T21:11:42.320442+0000","last_became_active":"2026-02-17T21:10:56.047458+0000","last_became_peered":"2026-02-17T21:10:56.047458+0000","last_unstale":"2026-02-17T21:11:42.320442+0000","last_undegraded":"2026-02-17T21:11:42.320442+0000","last_fullsized":"2026-02-17T21:11:42.320442+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2026-02-17T21:11:42.320415+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2026-02-17T21:11:42.320415+0000","last_clean_scrub_stamp":"2026-02-17T21:11:42.320415+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T03:00:41.231555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035170500000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'3","reported_seq":33,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:39.859807+0000","last_change":"2026-02-17T21:11:39.859817+0000","last_active":"2026-02-17T21:11:39.859807+0000","last_peered":"2026-02-17T21:11:39.859807+0000","last_clean":"2026-02-17T21:11:39.859807+0000","last_became_active":"2026-02-17T21:10:56.047333+0000","last_became_peered":"2026-02-17T21:10:56.047333+0000","last_unstale":"2026-02-17T21:11:39.859807+0000","last_undegraded":"2026-02-17T21:11:39.859807+0000","last_fullsized":"2026-02-17T21:11:39.859807+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'3","last_scrub_stamp":"2026-02-17T21:11:39.859784+0000","last_deep_scrub":"14'3","last_deep_scrub_stamp":"2026-02-17T21:11:39.859784+0000","last_clean_scrub_stamp":"2026-02-17T21:11:39.859784+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-19T01:15:24.863586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050980000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:40.645313+0000","last_change":"2026-02-17T21:11:40.645325+0000","last_active":"2026-02-17T21:11:40.645313+0000","last_peered":"2026-02-17T21:11:40.645313+0000","last_clean":"2026-02-17T21:11:40.645313+0000","last_became_active":"2026-02-17T21:10:56.047674+0000","last_became_peered":"2026-02-17T21:10:56.047674+0000","last_unstale":"2026-02-17T21:11:40.645313+0000","last_undegraded":"2026-02-17T21:11:40.645313+0000","last_fullsized":"2026-02-17T21:11:40.645313+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:11:40.645279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:11:40.645279+0000","last_clean_scrub_stamp":"2026-02-17T21:11:40.645279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T22:55:46.419411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00020513199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-17T21:11:43.367692+0000","last_change":"2026-02-17T21:11:43.367702+0000","last_active":"2026-02-17T21:11:43.367692+0000","last_peered":"2026-02-17T21:11:43.367692+0000","last_clean":"2026-02-17T21:11:43.367692+0000","last_became_active":"2026-02-17T21:10:56.048012+0000","last_became_peered":"2026-02-17T21:10:56.048012+0000","last_unstale":"2026-02-17T21:11:43.367692+0000","last_undegraded":"2026-02-17T21:11:43.367692+0000","last_fullsized":"2026-02-17T21:11:43.367692+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T21:11:43.367651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T21:11:43.367651+0000","last_clean_scrub_stamp":"2026-02-17T21:11:43.367651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-18T22:53:51.071794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035818699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705681,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27084,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830516,"statfs":{"total":107374182400,"available":107346448384,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738385,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27080,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830520,"statfs":{"total":107374182400,"available":107346452480,"internally_reserved":0,"allocated":122880,"data_stored":32696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738385,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27084,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830516,"statfs":{"total":107374182400,"available":107346448384,"internally_reserved":0,"allocated":126976,"data_stored":32715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T21:12:00.375 DEBUG:teuthology.orchestra.run.trial060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-17T21:12:00.546 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-17T21:12:00.546 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-17T21:12:00.546 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-17T21:12:00.547 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T21:12:00.583 INFO:tasks.ceph.osd.0:Stopped 2026-02-17T21:12:00.583 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-17T21:12:00.583 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T21:12:00.647 INFO:tasks.ceph.osd.1:Stopped 2026-02-17T21:12:00.648 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-17T21:12:00.648 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T21:12:00.715 INFO:tasks.ceph.osd.2:Stopped 2026-02-17T21:12:00.715 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-17T21:12:00.716 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-17T21:12:00.716 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T21:12:00.786 INFO:tasks.ceph.mgr.x:Stopped 2026-02-17T21:12:00.786 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-17T21:12:00.786 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T21:12:00.786 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T21:12:00.835 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T21:12:00.835 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-17T21:12:00.835 DEBUG:teuthology.orchestra.run.trial060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-17T21:12:00.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:12:00.902 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-17T21:12:00.969 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:12:00.969 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-17T21:12:01.035 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:12:01.035 DEBUG:teuthology.orchestra.run.trial060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-17T21:12:01.110 INFO:tasks.ceph:Archiving mon data... 2026-02-17T21:12:01.112 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889/data/mon.a.tgz 2026-02-17T21:12:01.112 DEBUG:teuthology.orchestra.run.trial060:> mktemp 2026-02-17T21:12:01.166 INFO:teuthology.orchestra.run.trial060.stdout:/tmp/tmp.Muip9Np9FS 2026-02-17T21:12:01.167 DEBUG:teuthology.orchestra.run.trial060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Muip9Np9FS 2026-02-17T21:12:01.248 DEBUG:teuthology.orchestra.run.trial060:> sudo chmod 0666 /tmp/tmp.Muip9Np9FS 2026-02-17T21:12:01.370 DEBUG:teuthology.orchestra.remote:trial060:/tmp/tmp.Muip9Np9FS is 131KB 2026-02-17T21:12:01.421 DEBUG:teuthology.orchestra.run.trial060:> rm -fr /tmp/tmp.Muip9Np9FS 2026-02-17T21:12:01.437 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-17T21:12:01.438 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T21:12:01.560 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-17T21:12:01.560 INFO:tasks.ceph:Archiving crash dumps... 2026-02-17T21:12:01.562 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889/remote/trial060/crash 2026-02-17T21:12:01.563 DEBUG:teuthology.orchestra.run.trial060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T21:12:01.623 INFO:tasks.ceph:Compressing logs... 2026-02-17T21:12:01.623 DEBUG:teuthology.orchestra.run.trial060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T21:12:01.702 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-17T21:12:01.702 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23898.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20037.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.23898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23898.log.gz 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26362.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph.tmp-client.admin.20037.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20037.log.gz 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26242.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26362.log.gz 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26121.log 2026-02-17T21:12:01.703 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32738.log 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26242.log.gz 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23549.log 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26121.log.gz 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32395.log 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32738.log.gz 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.23549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32199.log 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23549.log.gz 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25823.log 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32395.log.gz 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25614.log 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32199.log.gz 2026-02-17T21:12:01.704 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32493.log 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25823.log.gz 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23851.log 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25614.log.gz 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32689.log 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32493.log.gz 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.23851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32542.log 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23851.log.gz 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32640.log 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32689.log.gz 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32542.log.gz 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32591.log 2026-02-17T21:12:01.705 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32444.log 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32640.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32591.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25709.log 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25684.log 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32444.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25709.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24025.log 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32248.log 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25684.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.24025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26411.log 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24025.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32248.log.gz 2026-02-17T21:12:01.706 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26411.log.gz 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25568.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25973.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr: 87.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25568.log.gz 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25973.log.gz 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25974.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25872.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25974.log.gz 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26241.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24089.log 2026-02-17T21:12:01.707 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25872.log.gz 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26241.log.gz 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32297.log 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.24089.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.24089.log.gz -5 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32785.log 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32297.log.gz 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26273.log 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32785.log.gz 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25975.log 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26273.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.32346.log 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26273.log.gz 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25975.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25975.log.gz 2026-02-17T21:12:01.708 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.32346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32346.log.gz 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23661.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26122.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.23661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25774.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23661.log.gz 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26122.log.gz 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23804.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.25774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25774.log.gz 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.23804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23804.log.gz 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26120.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26460.log 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr: 83.5% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T21:12:01.709 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26120.log.gz 2026-02-17T21:12:01.710 INFO:teuthology.orchestra.run.trial060.stderr:/var/log/ceph/ceph-client.admin.26460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26460.log.gz 2026-02-17T21:12:01.710 INFO:teuthology.orchestra.run.trial060.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-17T21:12:01.710 INFO:teuthology.orchestra.run.trial060.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-17T21:12:01.710 INFO:teuthology.orchestra.run.trial060.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-17T21:12:01.711 INFO:teuthology.orchestra.run.trial060.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-17T21:12:01.711 INFO:teuthology.orchestra.run.trial060.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-17T21:12:01.712 INFO:teuthology.orchestra.run.trial060.stderr: 2026-02-17T21:12:01.712 INFO:teuthology.orchestra.run.trial060.stderr:real 0m0.032s 2026-02-17T21:12:01.712 INFO:teuthology.orchestra.run.trial060.stderr:user 0m0.085s 2026-02-17T21:12:01.712 INFO:teuthology.orchestra.run.trial060.stderr:sys 0m0.028s 2026-02-17T21:12:01.712 INFO:tasks.ceph:Archiving logs... 2026-02-17T21:12:01.713 DEBUG:teuthology.misc:Transferring archived files from trial060:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889/remote/trial060/log 2026-02-17T21:12:01.714 DEBUG:teuthology.orchestra.run.trial060:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T21:12:01.805 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T21:12:01.817 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T21:12:01.817 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T21:12:01.910 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T21:12:01.922 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T21:12:01.923 DEBUG:teuthology.orchestra.run.trial060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T21:12:01.938 INFO:teuthology.orchestra.run.trial060.stderr:bash: line 1: ntpq: command not found 2026-02-17T21:12:01.958 INFO:teuthology.orchestra.run.trial060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T21:12:01.958 INFO:teuthology.orchestra.run.trial060.stdout:=============================================================================== 2026-02-17T21:12:01.958 INFO:teuthology.orchestra.run.trial060.stdout:^- 198-23-133-146-host.colo> 3 6 77 47 -176us[ -176us] +/- 77ms 2026-02-17T21:12:01.958 INFO:teuthology.orchestra.run.trial060.stdout:^* time.cloudflare.com 3 6 77 48 +11us[ +150us] +/- 7279us 2026-02-17T21:12:01.958 INFO:teuthology.orchestra.run.trial060.stdout:^- ntp3.radio-sunshine.org 2 6 77 49 -655us[ -516us] +/- 76ms 2026-02-17T21:12:01.958 INFO:teuthology.orchestra.run.trial060.stdout:^- 72-14-182-49.ip.linodeus> 2 6 77 48 +2194us[+2333us] +/- 79ms 2026-02-17T21:12:01.959 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T21:12:01.970 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T21:12:01.971 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T21:12:01.987 DEBUG:teuthology.orchestra.run.trial060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T21:12:02.039 DEBUG:teuthology.orchestra.run.trial060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-17T21:12:02.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:12:02.104 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T21:12:02.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T21:12:02.132 INFO:teuthology.task.internal:Duration was 256.457402 seconds 2026-02-17T21:12:02.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T21:12:02.145 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T21:12:02.145 DEBUG:teuthology.orchestra.run.trial060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T21:12:02.182 INFO:teuthology.orchestra.run.trial060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T21:12:02.315 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T21:12:02.315 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial060.front.sepia.ceph.com 2026-02-17T21:12:02.316 DEBUG:teuthology.orchestra.run.trial060:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T21:12:02.373 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T21:12:02.373 DEBUG:teuthology.orchestra.run.trial060:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T21:12:02.455 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T21:12:02.455 DEBUG:teuthology.orchestra.run.trial060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T21:12:02.517 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T21:12:02.519 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T21:12:02.519 INFO:teuthology.orchestra.run.trial060.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T21:12:02.519 INFO:teuthology.orchestra.run.trial060.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T21:12:02.519 INFO:teuthology.orchestra.run.trial060.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T21:12:02.522 INFO:teuthology.orchestra.run.trial060.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T21:12:02.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T21:12:02.540 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T21:12:02.540 DEBUG:teuthology.orchestra.run.trial060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T21:12:02.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T21:12:02.615 DEBUG:teuthology.orchestra.run.trial060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:12:02.652 INFO:teuthology.orchestra.run.trial060.stdout:kernel.core_pattern = core 2026-02-17T21:12:02.667 DEBUG:teuthology.orchestra.run.trial060:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:12:02.722 WARNING:teuthology.task.internal:Found coredumps on ubuntu@trial060.front.sepia.ceph.com, flagging run as failed 2026-02-17T21:12:02.723 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T21:12:02.739 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T21:12:02.740 DEBUG:teuthology.misc:Transferring archived files from trial060:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889/remote/trial060 2026-02-17T21:12:02.741 DEBUG:teuthology.orchestra.run.trial060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T21:12:05.087 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2026-02-17T21:12:05.091 INFO:teuthology.task.internal: core looks like: /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53889/remote/trial060/coredump/1771362697.31925.core.gz: gzip compressed data, was "1771362697.31925.core", last modified: Tue Feb 17 21:12:02 2026, max compression, original size modulo 2^32 128684032 2026-02-17T21:12:05.091 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2026-02-17T21:12:05.252 INFO:teuthology.task.internal: core looks like: /tmp/tmp0app913z: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-kvstore-tool bluestore-kv ./cephtool.NM8 destructive-repair', real uid: 1000, effective uid: 1000, real gid: 1002, effective gid: 1002, execfn: '/usr/bin/ceph-kvstore-tool', platform: 'x86_64' 2026-02-17T21:12:05.254 INFO:teuthology.task.internal: dump_program: ceph-kvstore-tool 2026-02-17T21:12:05.254 DEBUG:teuthology.orchestra.run.trial060:> which ceph-kvstore-tool 2026-02-17T21:12:05.273 INFO:teuthology.orchestra.run.trial060.stdout:/usr/bin/ceph-kvstore-tool 2026-02-17T21:12:05.330 DEBUG:teuthology.orchestra.remote:trial060:/usr/bin/ceph-kvstore-tool is 10MB 2026-02-17T21:12:05.459 DEBUG:teuthology.orchestra.run.trial060:> mktemp 2026-02-17T21:12:05.475 INFO:teuthology.orchestra.run.trial060.stdout:/tmp/tmp.a5ivtfv0Rn 2026-02-17T21:12:05.476 DEBUG:teuthology.orchestra.run.trial060:> sudo cp /usr/bin/ceph-kvstore-tool.debug /tmp/tmp.a5ivtfv0Rn 2026-02-17T21:12:05.539 INFO:teuthology.orchestra.run.trial060.stderr:cp: cannot stat '/usr/bin/ceph-kvstore-tool.debug': No such file or directory 2026-02-17T21:12:05.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:12:05.540 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 421, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 364, in fetch_binaries_for_coredumps remote.get_file(debug_path, coredump_path) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 674, in get_file self.run(args=args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial060 with status 1: 'sudo cp /usr/bin/ceph-kvstore-tool.debug /tmp/tmp.a5ivtfv0Rn' 2026-02-17T21:12:05.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T21:12:05.558 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T21:12:05.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T21:12:05.574 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T21:12:05.574 DEBUG:teuthology.orchestra.run.trial060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 792232 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Feb 17 21:12 /home/ubuntu/cephtest 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 792244 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Feb 17 21:12 /home/ubuntu/cephtest/archive 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 792298 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 21:12 /home/ubuntu/cephtest/archive/syslog 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 786976 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Feb 17 21:07 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 786980 56 -rw-r--r-- 1 ubuntu ubuntu 55858 Feb 17 21:12 /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 786975 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Feb 17 21:07 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 786881 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 21:12 /home/ubuntu/cephtest/archive/audit 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 786972 52 -rw------- 1 ubuntu ubuntu 50181 Feb 17 21:12 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2026-02-17T21:12:05.597 INFO:teuthology.orchestra.run.trial060.stdout: 792247 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 21:11 /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:12:05.598 INFO:teuthology.orchestra.run.trial060.stdout: 804355 18520 -rw------- 1 ubuntu ubuntu 128684032 Feb 17 21:11 /home/ubuntu/cephtest/archive/coredump/1771362697.31925.core 2026-02-17T21:12:05.598 INFO:teuthology.orchestra.run.trial060.stdout: 804274 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 21:11 /home/ubuntu/cephtest/mnt.0 2026-02-17T21:12:05.598 INFO:teuthology.orchestra.run.trial060.stdout: 804275 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 21:11 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T21:12:05.598 INFO:teuthology.orchestra.run.trial060.stdout: 1063246 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 21:11 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T21:12:05.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:12:05.599 INFO:teuthology.orchestra.run.trial060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-17T21:12:05.600 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 421, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 364, in fetch_binaries_for_coredumps remote.get_file(debug_path, coredump_path) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 674, in get_file self.run(args=args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial060 with status 1: 'sudo cp /usr/bin/ceph-kvstore-tool.debug /tmp/tmp.a5ivtfv0Rn' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial060 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T21:12:05.603 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T21:12:05.620 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T21:12:05.639 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial060 with status 1: 'sudo cp /usr/bin/ceph-kvstore-tool.debug /tmp/tmp.a5ivtfv0Rn' 2026-02-17T21:12:05.645 INFO:teuthology.run:Summary data: description: rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_latest}} duration: 256.4574019908905 failure_reason: 'Command failed (workunit test cephtool/test_kvstore_tool.sh) on trial060 with status 139: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-17T21:12:05.645 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T21:12:05.771 INFO:teuthology.run:FAIL