2026-02-21T11:53:21.976 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T11:53:21.991 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T11:53:22.069 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613 branch: wip-bharath21-testing-2026-02-20-1039 description: rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '62613' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial no_nested_subset: false os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath21-testing-2026-02-20-1039 ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 30 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 workunit: branch: wip-bharath21-testing-2026-02-20-1039 sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - osd.4 - osd.5 - osd.6 - osd.7 - osd.8 - osd.9 - client.0 seed: 6481 sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath21-testing-2026-02-20-1039 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 targets: trial196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCPx2kFO+79NdagzaohMP3VPxz2h0IJWDkP5cc9PhCZ4/EJweAIx7fdG3wshh7bjsNx5sJmAujEWKAOkQDD0CeQ= tasks: - install: null - ceph: conf: osd: osd max object name len: 400 osd max object namespace len: 64 log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - workunit: clients: all: - rados/test_health_warnings.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_10:47:12 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T11:53:22.070 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa; will attempt to use it 2026-02-21T11:53:22.070 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa/tasks 2026-02-21T11:53:22.070 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T11:53:22.072 INFO:teuthology.task.internal:Checking packages... 2026-02-21T11:53:22.072 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f7273f3049146bdd7fa7b4a400114872a50228e9' 2026-02-21T11:53:22.072 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T11:53:22.072 INFO:teuthology.packaging:ref: None 2026-02-21T11:53:22.072 INFO:teuthology.packaging:tag: None 2026-02-21T11:53:22.072 INFO:teuthology.packaging:branch: wip-bharath21-testing-2026-02-20-1039 2026-02-21T11:53:22.072 INFO:teuthology.packaging:sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:53:22.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath21-testing-2026-02-20-1039 2026-02-21T11:53:22.190 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:53:22.191 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T11:53:22.203 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T11:53:22.203 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T11:53:22.216 INFO:teuthology.task.internal:Saving configuration 2026-02-21T11:53:22.225 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T11:53:22.238 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T11:53:22.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial196.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 11:49:52.530554', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICVsurPTqG7o6bgFtbpUEBqpk8DUBVllcdinxO+ck2c7'} 2026-02-21T11:53:22.316 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T11:53:22.328 INFO:teuthology.task.internal:roles: ubuntu@trial196.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0'] 2026-02-21T11:53:22.328 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T11:53:22.417 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7818220ff060>, signals=[15]) 2026-02-21T11:53:22.417 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T11:53:22.431 INFO:teuthology.task.internal:Opening connections... 2026-02-21T11:53:22.431 DEBUG:teuthology.task.internal:connecting to ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:53:22.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T11:53:22.492 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T11:53:22.503 DEBUG:teuthology.orchestra.run.trial196:> uname -m 2026-02-21T11:53:22.571 INFO:teuthology.orchestra.run.trial196.stdout:x86_64 2026-02-21T11:53:22.571 DEBUG:teuthology.orchestra.run.trial196:> cat /etc/os-release 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:NAME="Ubuntu" 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:VERSION_ID="22.04" 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:VERSION_CODENAME=jammy 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:ID=ubuntu 2026-02-21T11:53:22.622 INFO:teuthology.orchestra.run.trial196.stdout:ID_LIKE=debian 2026-02-21T11:53:22.623 INFO:teuthology.orchestra.run.trial196.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-21T11:53:22.623 INFO:teuthology.orchestra.run.trial196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-21T11:53:22.623 INFO:teuthology.orchestra.run.trial196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-21T11:53:22.623 INFO:teuthology.orchestra.run.trial196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-21T11:53:22.623 INFO:teuthology.orchestra.run.trial196.stdout:UBUNTU_CODENAME=jammy 2026-02-21T11:53:22.623 INFO:teuthology.lock.ops:Updating trial196.front.sepia.ceph.com on lock server 2026-02-21T11:53:22.715 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T11:53:22.734 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T11:53:22.744 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T11:53:22.744 DEBUG:teuthology.orchestra.run.trial196:> test '!' -e /home/ubuntu/cephtest 2026-02-21T11:53:22.751 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T11:53:22.765 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T11:53:22.765 DEBUG:teuthology.orchestra.run.trial196:> test -z $(ls -A /var/lib/ceph) 2026-02-21T11:53:22.806 INFO:teuthology.orchestra.run.trial196.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-21T11:53:22.806 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T11:53:22.884 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T11:53:22.905 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-21T11:53:22.905 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-21T11:53:22.905 DEBUG:teuthology.orchestra.run.trial196:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T11:53:22.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:53:22.908 DEBUG:teuthology.orchestra.run.trial196:> uname -r 2026-02-21T11:53:22.954 INFO:teuthology.orchestra.run.trial196.stdout:5.15.0-170-generic 2026-02-21T11:53:22.954 INFO:teuthology.task.kernel:Running kernel on trial196: 5.15.0-170-generic 2026-02-21T11:53:22.954 DEBUG:teuthology.orchestra.run.trial196:> sudo apt-get clean 2026-02-21T11:53:23.012 DEBUG:teuthology.orchestra.run.trial196:> sudo apt-get update 2026-02-21T11:53:23.545 INFO:teuthology.orchestra.run.trial196.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T11:53:23.618 INFO:teuthology.orchestra.run.trial196.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-21T11:53:23.898 INFO:teuthology.orchestra.run.trial196.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-21T11:53:24.019 INFO:teuthology.orchestra.run.trial196.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-21T11:53:24.134 INFO:teuthology.orchestra.run.trial196.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-21T11:53:24.463 INFO:teuthology.orchestra.run.trial196.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-21T11:53:24.472 INFO:teuthology.orchestra.run.trial196.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-21T11:53:24.472 INFO:teuthology.orchestra.run.trial196.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-21T11:53:24.501 INFO:teuthology.orchestra.run.trial196.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-21T11:53:24.507 INFO:teuthology.orchestra.run.trial196.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-21T11:53:24.508 INFO:teuthology.orchestra.run.trial196.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-21T11:53:24.636 INFO:teuthology.orchestra.run.trial196.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-21T11:53:24.655 INFO:teuthology.orchestra.run.trial196.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-21T11:53:24.726 INFO:teuthology.orchestra.run.trial196.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-21T11:53:24.734 INFO:teuthology.orchestra.run.trial196.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-21T11:53:24.734 INFO:teuthology.orchestra.run.trial196.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-21T11:53:24.761 INFO:teuthology.orchestra.run.trial196.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-21T11:53:24.762 INFO:teuthology.orchestra.run.trial196.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-21T11:53:24.884 INFO:teuthology.orchestra.run.trial196.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-21T11:53:28.467 INFO:teuthology.orchestra.run.trial196.stdout:Fetched 22.6 MB in 3s (8243 kB/s) 2026-02-21T11:53:28.986 INFO:teuthology.orchestra.run.trial196.stdout:Reading package lists... 2026-02-21T11:53:28.999 DEBUG:teuthology.orchestra.run.trial196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-21T11:53:29.032 INFO:teuthology.orchestra.run.trial196.stdout:Reading package lists... 2026-02-21T11:53:29.139 INFO:teuthology.orchestra.run.trial196.stdout:Building dependency tree... 2026-02-21T11:53:29.139 INFO:teuthology.orchestra.run.trial196.stdout:Reading state information... 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout:linux-image-generic set to manually installed. 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout: sg3-utils-udev 2026-02-21T11:53:29.227 INFO:teuthology.orchestra.run.trial196.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T11:53:29.246 INFO:teuthology.orchestra.run.trial196.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T11:53:29.248 DEBUG:teuthology.orchestra.run.trial196:> dpkg -s linux-image-generic 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Package: linux-image-generic 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Status: install ok installed 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Priority: optional 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Section: kernel 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Installed-Size: 21 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Maintainer: Ubuntu Kernel Team 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Architecture: amd64 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Source: linux-meta 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Version: 5.15.0.170.159 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Recommends: thermald 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout:Description: Generic Linux kernel image 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout: This package will always depend on the latest generic kernel image 2026-02-21T11:53:29.304 INFO:teuthology.orchestra.run.trial196.stdout: available. 2026-02-21T11:53:29.304 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T11:53:29.304 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T11:53:29.305 DEBUG:teuthology.orchestra.run.trial196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T11:53:29.355 INFO:teuthology.orchestra.run.trial196.stdout:ttyS1 2026-02-21T11:53:29.377 DEBUG:teuthology.parallel:result is None 2026-02-21T11:53:29.377 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T11:53:29.393 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T11:53:29.393 DEBUG:teuthology.orchestra.run.trial196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T11:53:29.423 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T11:53:29.436 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T11:53:29.449 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T11:53:29.449 DEBUG:teuthology.orchestra.run.trial196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T11:53:29.474 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T11:53:29.489 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T11:53:29.489 DEBUG:teuthology.orchestra.run.trial196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T11:53:29.520 INFO:teuthology.orchestra.run.trial196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:53:29.526 INFO:teuthology.orchestra.run.trial196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:53:29.527 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T11:53:29.539 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T11:53:29.539 DEBUG:teuthology.orchestra.run.trial196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T11:53:29.583 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T11:53:29.607 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T11:53:29.607 DEBUG:teuthology.orchestra.run.trial196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T11:53:29.631 DEBUG:teuthology.orchestra.run.trial196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T11:53:29.680 DEBUG:teuthology.orchestra.run.trial196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T11:53:29.726 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:53:29.727 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T11:53:29.775 DEBUG:teuthology.orchestra.run.trial196:> sudo service rsyslog restart 2026-02-21T11:53:29.836 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T11:53:29.847 INFO:teuthology.task.internal:Starting timer... 2026-02-21T11:53:29.847 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T11:53:29.861 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T11:53:29.872 DEBUG:teuthology.task.selinux:Excluding trial196: OS 'ubuntu' does not support SELinux 2026-02-21T11:53:29.872 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T11:53:29.872 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-21T11:53:29.872 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T11:53:29.872 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T11:53:29.884 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T11:53:29.891 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T11:53:29.892 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T11:55:48.453 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial196.front.sepia.ceph.com')] 2026-02-21T11:55:48.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial196.front.sepia.ceph.com' 2026-02-21T11:55:48.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T11:55:48.525 DEBUG:teuthology.orchestra.run.trial196:> true 2026-02-21T11:55:48.775 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial196.front.sepia.ceph.com' 2026-02-21T11:55:48.775 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T11:55:48.790 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T11:55:48.790 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T11:55:48.790 DEBUG:teuthology.orchestra.run.trial196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Command line: ntpd -gq 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: ---------------------------------------------------- 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: ntp-4 is maintained by Network Time Foundation, 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: corporation. Support and training for ntp-4 are 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: available at https://www.nwtime.org/support 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: ---------------------------------------------------- 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: proto: precision = 0.030 usec (-25) 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: basedate set to 2022-02-04 2026-02-21T11:55:48.833 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: gps base set to 2022-02-06 (week 2196) 2026-02-21T11:55:48.834 INFO:teuthology.orchestra.run.trial196.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T11:55:48.834 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T11:55:48.834 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: restrict ::: KOD does nothing without LIMITED. 2026-02-21T11:55:48.834 INFO:teuthology.orchestra.run.trial196.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listen and drop on 0 v6wildcard [::]:123 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listen normally on 2 lo 127.0.0.1:123 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listen normally on 3 enp1s0f0 10.20.193.196:123 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listen normally on 4 lo [::1]:123 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:61ea%2]:123 2026-02-21T11:55:48.835 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:48 ntpd[9421]: Listening on routing socket on fd #22 for interface updates 2026-02-21T11:55:49.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:49 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:49.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:49 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:49.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:49 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:49.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:49 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:49.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:49 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:49.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:49 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:51.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:51.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:51.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:51.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:51.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:51.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:51.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:51.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:51 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:51.903 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:52 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:51.903 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:52 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:51.903 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:52 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:51.903 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:52 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:52.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:52 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:52.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:52 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:52.915 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:52.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:52.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:52.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:53.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:53.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:53.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:53.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:53.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:53.834 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:53.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:53.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:53.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:53.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:53.870 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:53.870 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:53.870 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:53.870 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:53 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:53.902 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:54 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:53.902 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:54 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:53.902 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:54 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:53.902 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:54 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:54.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:54 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:54.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:54 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:54.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:54.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:54.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:54.916 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:55.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:55.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:55.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:55.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:55.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:55.833 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:55.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:55:55.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:55:55.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:55:55.842 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:55:55.842 INFO:teuthology.orchestra.run.trial196.stdout:21 Feb 11:55:55 ntpd[9421]: ntpd: time set -0.112778 s 2026-02-21T11:55:55.842 INFO:teuthology.orchestra.run.trial196.stdout:ntpd: time set -0.112778s 2026-02-21T11:55:55.843 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-21T11:55:55.843 INFO:teuthology.orchestra.run.trial196.stderr:21 Feb 11:55:55 ntpd[9421]: can't open /var/log/ntpstats/loopstats.20260221: Permission denied 2026-02-21T11:55:55.867 INFO:teuthology.orchestra.run.trial196.stdout:No association ID's returned 2026-02-21T11:55:55.869 INFO:teuthology.run_tasks:Running task install... 2026-02-21T11:55:55.880 DEBUG:teuthology.task.install:project ceph 2026-02-21T11:55:55.880 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'}} 2026-02-21T11:55:55.880 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'} 2026-02-21T11:55:55.880 INFO:teuthology.task.install:Using flavor: default 2026-02-21T11:55:55.884 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T11:55:55.884 INFO:teuthology.task.install:extra packages: [] 2026-02-21T11:55:55.885 DEBUG:teuthology.orchestra.run.trial196:> sudo apt-key list | grep Ceph 2026-02-21T11:55:55.957 INFO:teuthology.orchestra.run.trial196.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-21T11:55:55.984 INFO:teuthology.orchestra.run.trial196.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-21T11:55:55.984 INFO:teuthology.orchestra.run.trial196.stdout:uid [ unknown] Ceph.com (release key) 2026-02-21T11:55:55.984 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-21T11:55:55.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:55:56.169 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default/ 2026-02-21T11:55:56.169 INFO:teuthology.task.install.deb:Package version is 20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:55:56.303 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:55:56.303 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-21T11:55:56.313 DEBUG:teuthology.orchestra.run.trial196:> sudo apt-get update 2026-02-21T11:55:56.477 INFO:teuthology.orchestra.run.trial196.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy InRelease 2026-02-21T11:55:56.488 INFO:teuthology.orchestra.run.trial196.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-21T11:55:56.501 INFO:teuthology.orchestra.run.trial196.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-21T11:55:56.513 INFO:teuthology.orchestra.run.trial196.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-21T11:55:56.820 INFO:teuthology.orchestra.run.trial196.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T11:55:56.901 INFO:teuthology.orchestra.run.trial196.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-21T11:55:56.989 INFO:teuthology.orchestra.run.trial196.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-21T11:55:57.077 INFO:teuthology.orchestra.run.trial196.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-21T11:55:57.225 INFO:teuthology.orchestra.run.trial196.stdout:Fetched 26.9 kB in 1s (35.9 kB/s) 2026-02-21T11:55:57.742 INFO:teuthology.orchestra.run.trial196.stdout:Reading package lists... 2026-02-21T11:55:57.755 DEBUG:teuthology.orchestra.run.trial196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5397-gf7273f30-1jammy cephadm=20.3.0-5397-gf7273f30-1jammy ceph-mds=20.3.0-5397-gf7273f30-1jammy ceph-mgr=20.3.0-5397-gf7273f30-1jammy ceph-osd=20.3.0-5397-gf7273f30-1jammy ceph-osd-classic=20.3.0-5397-gf7273f30-1jammy ceph-common=20.3.0-5397-gf7273f30-1jammy ceph-fuse=20.3.0-5397-gf7273f30-1jammy ceph-test=20.3.0-5397-gf7273f30-1jammy ceph-volume=20.3.0-5397-gf7273f30-1jammy radosgw=20.3.0-5397-gf7273f30-1jammy python3-rados=20.3.0-5397-gf7273f30-1jammy python3-rgw=20.3.0-5397-gf7273f30-1jammy python3-cephfs=20.3.0-5397-gf7273f30-1jammy python3-rbd=20.3.0-5397-gf7273f30-1jammy libcephfs2=20.3.0-5397-gf7273f30-1jammy libcephfs-dev=20.3.0-5397-gf7273f30-1jammy librados2=20.3.0-5397-gf7273f30-1jammy librbd1=20.3.0-5397-gf7273f30-1jammy rbd-fuse=20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:55:57.789 INFO:teuthology.orchestra.run.trial196.stdout:Reading package lists... 2026-02-21T11:55:57.898 INFO:teuthology.orchestra.run.trial196.stdout:Building dependency tree... 2026-02-21T11:55:57.898 INFO:teuthology.orchestra.run.trial196.stdout:Reading state information... 2026-02-21T11:55:57.990 INFO:teuthology.orchestra.run.trial196.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T11:55:57.990 INFO:teuthology.orchestra.run.trial196.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T11:55:57.990 INFO:teuthology.orchestra.run.trial196.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout: sg3-utils-udev 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout:The following additional packages will be installed: 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T11:55:57.991 INFO:teuthology.orchestra.run.trial196.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout:Suggested packages: 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout:Recommended packages: 2026-02-21T11:55:57.992 INFO:teuthology.orchestra.run.trial196.stdout: btrfs-tools 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout:The following NEW packages will be installed: 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-21T11:55:58.023 INFO:teuthology.orchestra.run.trial196.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-21T11:55:58.024 INFO:teuthology.orchestra.run.trial196.stdout: socat xmlstarlet 2026-02-21T11:55:58.103 INFO:teuthology.orchestra.run.trial196.stdout:0 upgraded, 87 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T11:55:58.103 INFO:teuthology.orchestra.run.trial196.stdout:Need to get 699 MB of archives. 2026-02-21T11:55:58.103 INFO:teuthology.orchestra.run.trial196.stdout:After this operation, 2661 MB of additional disk space will be used. 2026-02-21T11:55:58.103 INFO:teuthology.orchestra.run.trial196.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5397-gf7273f30-1jammy [3794 kB] 2026-02-21T11:55:58.128 INFO:teuthology.orchestra.run.trial196.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-21T11:55:58.174 INFO:teuthology.orchestra.run.trial196.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-21T11:55:58.175 INFO:teuthology.orchestra.run.trial196.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-21T11:55:58.184 INFO:teuthology.orchestra.run.trial196.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-21T11:55:58.211 INFO:teuthology.orchestra.run.trial196.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5397-gf7273f30-1jammy [2910 kB] 2026-02-21T11:55:58.213 INFO:teuthology.orchestra.run.trial196.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-21T11:55:58.214 INFO:teuthology.orchestra.run.trial196.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-21T11:55:58.220 INFO:teuthology.orchestra.run.trial196.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-21T11:55:58.222 INFO:teuthology.orchestra.run.trial196.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-21T11:55:58.222 INFO:teuthology.orchestra.run.trial196.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-21T11:55:58.223 INFO:teuthology.orchestra.run.trial196.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-21T11:55:58.223 INFO:teuthology.orchestra.run.trial196.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-21T11:55:58.227 INFO:teuthology.orchestra.run.trial196.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-21T11:55:58.228 INFO:teuthology.orchestra.run.trial196.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-21T11:55:58.228 INFO:teuthology.orchestra.run.trial196.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-21T11:55:58.234 INFO:teuthology.orchestra.run.trial196.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-21T11:55:58.234 INFO:teuthology.orchestra.run.trial196.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-21T11:55:58.234 INFO:teuthology.orchestra.run.trial196.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-21T11:55:58.234 INFO:teuthology.orchestra.run.trial196.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-21T11:55:58.235 INFO:teuthology.orchestra.run.trial196.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-21T11:55:58.243 INFO:teuthology.orchestra.run.trial196.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-21T11:55:58.243 INFO:teuthology.orchestra.run.trial196.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-21T11:55:58.245 INFO:teuthology.orchestra.run.trial196.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-21T11:55:58.245 INFO:teuthology.orchestra.run.trial196.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-21T11:55:58.246 INFO:teuthology.orchestra.run.trial196.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-21T11:55:58.249 INFO:teuthology.orchestra.run.trial196.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5397-gf7273f30-1jammy [882 kB] 2026-02-21T11:55:58.252 INFO:teuthology.orchestra.run.trial196.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-21T11:55:58.253 INFO:teuthology.orchestra.run.trial196.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-21T11:55:58.253 INFO:teuthology.orchestra.run.trial196.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-21T11:55:58.264 INFO:teuthology.orchestra.run.trial196.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5397-gf7273f30-1jammy [359 kB] 2026-02-21T11:55:58.267 INFO:teuthology.orchestra.run.trial196.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-21T11:55:58.269 INFO:teuthology.orchestra.run.trial196.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-21T11:55:58.269 INFO:teuthology.orchestra.run.trial196.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5397-gf7273f30-1jammy [32.8 kB] 2026-02-21T11:55:58.269 INFO:teuthology.orchestra.run.trial196.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-21T11:55:58.270 INFO:teuthology.orchestra.run.trial196.stdout:Get:36 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5397-gf7273f30-1jammy [225 kB] 2026-02-21T11:55:58.273 INFO:teuthology.orchestra.run.trial196.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5397-gf7273f30-1jammy [88.4 kB] 2026-02-21T11:55:58.275 INFO:teuthology.orchestra.run.trial196.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5397-gf7273f30-1jammy [338 kB] 2026-02-21T11:55:58.279 INFO:teuthology.orchestra.run.trial196.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-21T11:55:58.280 INFO:teuthology.orchestra.run.trial196.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-21T11:55:58.280 INFO:teuthology.orchestra.run.trial196.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-21T11:55:58.280 INFO:teuthology.orchestra.run.trial196.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-21T11:55:58.280 INFO:teuthology.orchestra.run.trial196.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-21T11:55:58.280 INFO:teuthology.orchestra.run.trial196.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-21T11:55:58.281 INFO:teuthology.orchestra.run.trial196.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5397-gf7273f30-1jammy [9089 kB] 2026-02-21T11:55:58.282 INFO:teuthology.orchestra.run.trial196.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-21T11:55:58.283 INFO:teuthology.orchestra.run.trial196.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-21T11:55:58.283 INFO:teuthology.orchestra.run.trial196.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-21T11:55:58.291 INFO:teuthology.orchestra.run.trial196.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-21T11:55:58.292 INFO:teuthology.orchestra.run.trial196.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-21T11:55:58.293 INFO:teuthology.orchestra.run.trial196.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-21T11:55:58.295 INFO:teuthology.orchestra.run.trial196.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-21T11:55:58.297 INFO:teuthology.orchestra.run.trial196.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-21T11:55:58.300 INFO:teuthology.orchestra.run.trial196.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-21T11:55:58.300 INFO:teuthology.orchestra.run.trial196.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-21T11:55:58.301 INFO:teuthology.orchestra.run.trial196.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-21T11:55:58.302 INFO:teuthology.orchestra.run.trial196.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-21T11:55:58.306 INFO:teuthology.orchestra.run.trial196.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-21T11:55:58.310 INFO:teuthology.orchestra.run.trial196.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-21T11:55:58.311 INFO:teuthology.orchestra.run.trial196.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-21T11:55:58.311 INFO:teuthology.orchestra.run.trial196.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-21T11:55:58.312 INFO:teuthology.orchestra.run.trial196.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-21T11:55:58.431 INFO:teuthology.orchestra.run.trial196.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5397-gf7273f30-1jammy [112 kB] 2026-02-21T11:55:58.432 INFO:teuthology.orchestra.run.trial196.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5397-gf7273f30-1jammy [261 kB] 2026-02-21T11:55:58.436 INFO:teuthology.orchestra.run.trial196.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5397-gf7273f30-1jammy [31.5 MB] 2026-02-21T11:55:58.958 INFO:teuthology.orchestra.run.trial196.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5397-gf7273f30-1jammy [4984 kB] 2026-02-21T11:55:59.034 INFO:teuthology.orchestra.run.trial196.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5397-gf7273f30-1jammy [294 kB] 2026-02-21T11:55:59.038 INFO:teuthology.orchestra.run.trial196.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5397-gf7273f30-1jammy [127 kB] 2026-02-21T11:55:59.040 INFO:teuthology.orchestra.run.trial196.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5397-gf7273f30-1jammy [934 kB] 2026-02-21T11:55:59.056 INFO:teuthology.orchestra.run.trial196.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5397-gf7273f30-1jammy [7538 kB] 2026-02-21T11:55:59.181 INFO:teuthology.orchestra.run.trial196.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5397-gf7273f30-1jammy [468 MB] 2026-02-21T11:56:07.132 INFO:teuthology.orchestra.run.trial196.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5397-gf7273f30-1jammy [180 kB] 2026-02-21T11:56:07.132 INFO:teuthology.orchestra.run.trial196.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5397-gf7273f30-1jammy [14.0 kB] 2026-02-21T11:56:07.133 INFO:teuthology.orchestra.run.trial196.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5397-gf7273f30-1jammy [1030 kB] 2026-02-21T11:56:07.138 INFO:teuthology.orchestra.run.trial196.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5397-gf7273f30-1jammy [2437 kB] 2026-02-21T11:56:07.151 INFO:teuthology.orchestra.run.trial196.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5397-gf7273f30-1jammy [1068 kB] 2026-02-21T11:56:07.156 INFO:teuthology.orchestra.run.trial196.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5397-gf7273f30-1jammy [190 kB] 2026-02-21T11:56:07.157 INFO:teuthology.orchestra.run.trial196.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5397-gf7273f30-1jammy [54.8 MB] 2026-02-21T11:56:08.053 INFO:teuthology.orchestra.run.trial196.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5397-gf7273f30-1jammy [8626 kB] 2026-02-21T11:56:08.187 INFO:teuthology.orchestra.run.trial196.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5397-gf7273f30-1jammy [14.2 kB] 2026-02-21T11:56:08.187 INFO:teuthology.orchestra.run.trial196.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5397-gf7273f30-1jammy [65.4 MB] 2026-02-21T11:56:09.267 INFO:teuthology.orchestra.run.trial196.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5397-gf7273f30-1jammy [137 kB] 2026-02-21T11:56:09.267 INFO:teuthology.orchestra.run.trial196.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5397-gf7273f30-1jammy [45.8 kB] 2026-02-21T11:56:09.268 INFO:teuthology.orchestra.run.trial196.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5397-gf7273f30-1jammy [32.4 kB] 2026-02-21T11:56:09.268 INFO:teuthology.orchestra.run.trial196.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5397-gf7273f30-1jammy [44.2 kB] 2026-02-21T11:56:09.268 INFO:teuthology.orchestra.run.trial196.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5397-gf7273f30-1jammy [18.3 MB] 2026-02-21T11:56:09.562 INFO:teuthology.orchestra.run.trial196.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5397-gf7273f30-1jammy [93.7 kB] 2026-02-21T11:56:09.841 INFO:teuthology.orchestra.run.trial196.stdout:Fetched 699 MB in 12s (60.6 MB/s) 2026-02-21T11:56:09.858 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-21T11:56:10.017 INFO:teuthology.orchestra.run.trial196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-21T11:56:10.019 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-21T11:56:10.021 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T11:56:10.039 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-21T11:56:10.051 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-21T11:56:10.051 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T11:56:10.070 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-21T11:56:10.082 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-21T11:56:10.083 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T11:56:10.101 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-21T11:56:10.110 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T11:56:10.115 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:56:10.147 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-21T11:56:10.154 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T11:56:10.155 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:56:10.171 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-21T11:56:10.179 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T11:56:10.179 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:56:10.199 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-21T11:56:10.207 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-21T11:56:10.207 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T11:56:10.222 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package librados2. 2026-02-21T11:56:10.230 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../07-librados2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.231 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking librados2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.273 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libnbd0. 2026-02-21T11:56:10.282 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-21T11:56:10.282 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-21T11:56:10.297 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package librbd1. 2026-02-21T11:56:10.305 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../09-librbd1_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.306 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking librbd1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.343 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libcephfs2. 2026-02-21T11:56:10.351 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.351 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libcephfs2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.370 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-rados. 2026-02-21T11:56:10.378 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.378 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-rados (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.396 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-21T11:56:10.404 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:10.404 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-ceph-argparse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.418 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-cephfs. 2026-02-21T11:56:10.426 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.426 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-cephfs (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.443 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-21T11:56:10.450 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:10.451 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.470 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-21T11:56:10.478 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-21T11:56:10.478 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T11:56:10.494 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-prettytable. 2026-02-21T11:56:10.502 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-21T11:56:10.502 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-21T11:56:10.517 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-rbd. 2026-02-21T11:56:10.524 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.525 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-rbd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.543 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-21T11:56:10.551 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-21T11:56:10.551 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T11:56:10.569 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package librgw2. 2026-02-21T11:56:10.577 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../19-librgw2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.578 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking librgw2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.675 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-rgw. 2026-02-21T11:56:10.683 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.683 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-rgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.699 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-21T11:56:10.707 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-21T11:56:10.707 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T11:56:10.722 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libradosstriper1. 2026-02-21T11:56:10.731 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.731 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libradosstriper1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:10.747 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-common. 2026-02-21T11:56:10.755 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:10.755 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:11.010 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-base. 2026-02-21T11:56:11.018 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:11.023 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-base (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:11.082 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-21T11:56:11.090 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-21T11:56:11.091 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-21T11:56:11.105 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-cheroot. 2026-02-21T11:56:11.114 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-21T11:56:11.114 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T11:56:11.131 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-21T11:56:11.139 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-21T11:56:11.140 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-21T11:56:11.154 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-21T11:56:11.163 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-21T11:56:11.163 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-21T11:56:11.177 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-21T11:56:11.185 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-21T11:56:11.185 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-21T11:56:11.200 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-tempora. 2026-02-21T11:56:11.208 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-21T11:56:11.209 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-21T11:56:11.223 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-portend. 2026-02-21T11:56:11.231 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-21T11:56:11.232 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-21T11:56:11.247 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-21T11:56:11.255 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-21T11:56:11.255 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-21T11:56:11.270 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-21T11:56:11.278 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-21T11:56:11.278 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-21T11:56:11.302 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-natsort. 2026-02-21T11:56:11.310 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-21T11:56:11.310 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-21T11:56:11.326 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-21T11:56:11.334 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:11.334 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:11.364 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-21T11:56:11.372 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:11.373 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:11.388 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mgr. 2026-02-21T11:56:11.396 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:11.397 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mgr (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:11.418 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mon. 2026-02-21T11:56:11.426 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:11.426 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:11.490 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-21T11:56:11.499 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-21T11:56:11.499 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T11:56:11.514 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-21T11:56:11.523 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:11.523 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-osd-classic (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.535 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-osd. 2026-02-21T11:56:15.548 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:15.548 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-osd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.568 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph. 2026-02-21T11:56:15.581 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../42-ceph_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:15.581 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.598 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-fuse. 2026-02-21T11:56:15.610 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:15.610 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.634 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mds. 2026-02-21T11:56:15.643 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:15.643 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mds (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.675 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package cephadm. 2026-02-21T11:56:15.684 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../45-cephadm_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:15.684 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.700 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-21T11:56:15.709 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T11:56:15.709 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T11:56:15.731 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-21T11:56:15.739 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:15.740 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:15.761 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-21T11:56:15.770 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-21T11:56:15.770 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-21T11:56:15.785 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-routes. 2026-02-21T11:56:15.793 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-21T11:56:15.793 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T11:56:15.813 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-21T11:56:15.821 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:15.822 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:16.513 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-21T11:56:16.523 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-21T11:56:16.523 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T11:56:16.562 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-joblib. 2026-02-21T11:56:16.570 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-21T11:56:16.571 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T11:56:16.598 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-21T11:56:16.606 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-21T11:56:16.607 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-21T11:56:16.621 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-sklearn. 2026-02-21T11:56:16.630 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-21T11:56:16.630 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T11:56:16.715 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-21T11:56:16.723 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:16.724 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:16.882 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-cachetools. 2026-02-21T11:56:16.890 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-21T11:56:16.891 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-21T11:56:16.906 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-rsa. 2026-02-21T11:56:16.914 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-21T11:56:16.914 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-21T11:56:16.934 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-google-auth. 2026-02-21T11:56:16.942 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-21T11:56:16.943 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-21T11:56:16.961 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-21T11:56:16.970 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-21T11:56:16.970 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T11:56:16.986 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-websocket. 2026-02-21T11:56:16.994 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-21T11:56:16.995 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-21T11:56:17.012 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-21T11:56:17.021 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-21T11:56:17.021 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T11:56:17.118 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-21T11:56:17.128 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:17.128 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:17.142 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-21T11:56:17.150 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-21T11:56:17.151 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T11:56:17.166 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-21T11:56:17.175 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T11:56:17.175 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T11:56:17.190 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package jq. 2026-02-21T11:56:17.199 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T11:56:17.199 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-21T11:56:17.213 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package socat. 2026-02-21T11:56:17.221 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-21T11:56:17.222 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-21T11:56:17.241 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package xmlstarlet. 2026-02-21T11:56:17.250 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-21T11:56:17.250 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-21T11:56:17.287 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-test. 2026-02-21T11:56:17.295 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:17.296 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-test (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:17.915 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package ceph-volume. 2026-02-21T11:56:17.923 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:56:17.924 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking ceph-volume (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:17.946 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-21T11:56:17.955 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:17.955 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libcephfs-daemon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:17.971 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-21T11:56:17.980 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:17.980 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libcephfs-proxy2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:17.995 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-21T11:56:18.004 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:18.004 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking libcephfs-dev (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:18.020 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package nvme-cli. 2026-02-21T11:56:18.029 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-21T11:56:18.029 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T11:56:18.062 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-21T11:56:18.070 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T11:56:18.070 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T11:56:18.100 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-21T11:56:18.109 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-21T11:56:18.109 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-21T11:56:18.125 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-pluggy. 2026-02-21T11:56:18.133 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-21T11:56:18.134 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-21T11:56:18.150 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-psutil. 2026-02-21T11:56:18.158 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-21T11:56:18.159 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-21T11:56:18.178 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-py. 2026-02-21T11:56:18.186 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-21T11:56:18.187 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-21T11:56:18.207 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-pygments. 2026-02-21T11:56:18.216 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-21T11:56:18.216 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T11:56:18.258 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-toml. 2026-02-21T11:56:18.266 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-21T11:56:18.267 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-21T11:56:18.283 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-pytest. 2026-02-21T11:56:18.291 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-21T11:56:18.292 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T11:56:18.313 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-simplejson. 2026-02-21T11:56:18.322 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-21T11:56:18.322 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-21T11:56:18.346 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package python3-webob. 2026-02-21T11:56:18.355 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-21T11:56:18.355 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T11:56:18.372 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-21T11:56:18.381 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-21T11:56:18.381 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-21T11:56:18.451 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package radosgw. 2026-02-21T11:56:18.460 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../85-radosgw_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:18.460 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking radosgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:18.637 INFO:teuthology.orchestra.run.trial196.stdout:Selecting previously unselected package rbd-fuse. 2026-02-21T11:56:18.646 INFO:teuthology.orchestra.run.trial196.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:56:18.646 INFO:teuthology.orchestra.run.trial196.stdout:Unpacking rbd-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:18.675 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-21T11:56:18.741 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T11:56:18.742 INFO:teuthology.orchestra.run.trial196.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T11:56:18.815 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T11:56:19.089 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-21T11:56:19.446 INFO:teuthology.orchestra.run.trial196.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-21T11:56:19.456 INFO:teuthology.orchestra.run.trial196.stdout:Setting up cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:19.502 INFO:teuthology.orchestra.run.trial196.stdout:Adding system user cephadm....done 2026-02-21T11:56:19.508 INFO:teuthology.orchestra.run.trial196.stdout:usermod: no changes 2026-02-21T11:56:19.514 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-21T11:56:19.586 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T11:56:19.587 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-21T11:56:19.661 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-21T11:56:19.737 INFO:teuthology.orchestra.run.trial196.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T11:56:19.738 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-21T11:56:19.834 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T11:56:19.962 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-21T11:56:20.036 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-21T11:56:20.111 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-ceph-argparse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:20.176 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T11:56:20.177 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-21T11:56:20.178 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T11:56:20.179 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T11:56:20.180 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-21T11:56:20.305 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-21T11:56:20.386 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libcephfs-proxy2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:20.387 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T11:56:20.466 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-21T11:56:20.554 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T11:56:20.815 INFO:teuthology.orchestra.run.trial196.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-21T11:56:20.816 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T11:56:20.913 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T11:56:21.042 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T11:56:21.126 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-21T11:56:21.202 INFO:teuthology.orchestra.run.trial196.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-21T11:56:21.203 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:21.296 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T11:56:21.816 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:56:21.821 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-21T11:56:21.897 INFO:teuthology.orchestra.run.trial196.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T11:56:21.898 INFO:teuthology.orchestra.run.trial196.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-21T11:56:21.899 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-21T11:56:21.972 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-21T11:56:22.045 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:56:22.046 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-21T11:56:22.123 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-21T11:56:22.199 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-21T11:56:22.279 INFO:teuthology.orchestra.run.trial196.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T11:56:22.280 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-21T11:56:22.364 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T11:56:22.365 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T11:56:22.441 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T11:56:22.526 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-21T11:56:22.601 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T11:56:22.602 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T11:56:22.740 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-21T11:56:22.815 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:56:22.816 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-21T11:56:22.898 INFO:teuthology.orchestra.run.trial196.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-21T11:56:22.899 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-21T11:56:23.029 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T11:56:23.030 INFO:teuthology.orchestra.run.trial196.stdout:Setting up librados2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.031 INFO:teuthology.orchestra.run.trial196.stdout:Setting up librgw2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.032 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.033 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T11:56:23.573 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libcephfs2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.574 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libradosstriper1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.575 INFO:teuthology.orchestra.run.trial196.stdout:Setting up librbd1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.576 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mgr-modules-core (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.577 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:23.651 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T11:56:23.651 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T11:56:24.039 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libcephfs-dev (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.039 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-rados (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.040 INFO:teuthology.orchestra.run.trial196.stdout:Setting up libcephfs-daemon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.041 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-rbd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.042 INFO:teuthology.orchestra.run.trial196.stdout:Setting up rbd-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.043 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-rgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.044 INFO:teuthology.orchestra.run.trial196.stdout:Setting up python3-cephfs (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.045 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.073 INFO:teuthology.orchestra.run.trial196.stdout:Adding group ceph....done 2026-02-21T11:56:24.109 INFO:teuthology.orchestra.run.trial196.stdout:Adding system user ceph....done 2026-02-21T11:56:24.119 INFO:teuthology.orchestra.run.trial196.stdout:Setting system user ceph properties....done 2026-02-21T11:56:24.125 INFO:teuthology.orchestra.run.trial196.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-21T11:56:24.214 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-21T11:56:24.486 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-21T11:56:24.858 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-test (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:24.858 INFO:teuthology.orchestra.run.trial196.stdout:Setting up radosgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:25.101 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T11:56:25.102 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T11:56:25.442 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-base (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:25.525 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-21T11:56:25.861 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mds (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:25.921 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T11:56:25.921 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T11:56:26.268 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mgr (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:26.348 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T11:56:26.348 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T11:56:26.686 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:26.687 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-osd-classic (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:26.704 INFO:teuthology.orchestra.run.trial196.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-21T11:56:26.714 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:26.725 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:26.798 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T11:56:26.798 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T11:56:27.183 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mgr-cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:27.195 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-mgr-dashboard (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:27.205 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-osd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:27.290 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T11:56:27.291 INFO:teuthology.orchestra.run.trial196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T11:56:27.669 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:27.670 INFO:teuthology.orchestra.run.trial196.stdout:Setting up ceph-volume (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:56:27.780 INFO:teuthology.orchestra.run.trial196.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-21T11:56:27.866 INFO:teuthology.orchestra.run.trial196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-21T11:56:28.539 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:28.539 INFO:teuthology.orchestra.run.trial196.stdout:Running kernel seems to be up-to-date. 2026-02-21T11:56:28.539 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout:The processor microcode seems to be up-to-date. 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout:No services need to be restarted. 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout:No containers need to be restarted. 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout:No user sessions are running outdated binaries. 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:28.540 INFO:teuthology.orchestra.run.trial196.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-21T11:56:29.380 INFO:teuthology.orchestra.run.trial196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-21T11:56:29.388 DEBUG:teuthology.parallel:result is None 2026-02-21T11:56:29.389 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:56:29.557 DEBUG:teuthology.orchestra.run.trial196:> dpkg-query -W -f '${Version}' ceph 2026-02-21T11:56:29.567 INFO:teuthology.orchestra.run.trial196.stdout:20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:56:29.568 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:56:29.568 INFO:teuthology.task.install:The correct ceph version 20.3.0-5397-gf7273f30-1jammy is installed. 2026-02-21T11:56:29.571 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T11:56:29.571 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:29.571 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T11:56:29.617 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T11:56:29.617 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:29.617 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T11:56:29.668 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T11:56:29.720 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T11:56:29.720 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:29.720 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T11:56:29.768 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T11:56:29.820 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T11:56:29.820 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:29.820 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T11:56:29.867 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T11:56:29.915 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T11:56:29.991 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T11:56:29.991 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod 777 /var/log/ceph 2026-02-21T11:56:29.999 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T11:56:29.999 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T11:56:30.047 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T11:56:30.047 DEBUG:teuthology.orchestra.run.trial196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T11:56:30.108 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T11:56:30.108 INFO:tasks.ceph:config {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T11:56:30.108 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613', 'branch': 'wip-bharath21-testing-2026-02-20-1039', 'description': 'rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '62613', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'trial', 'name': 'skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial', 'no_nested_subset': False, 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath21-testing-2026-02-20-1039'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'}}, 'workunit': {'branch': 'wip-bharath21-testing-2026-02-20-1039', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'}}, 'owner': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'priority': 60, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0']], 'seed': 6481, 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath21-testing-2026-02-20-1039', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9', 'targets': {'trial196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCPx2kFO+79NdagzaohMP3VPxz2h0IJWDkP5cc9PhCZ4/EJweAIx7fdG3wshh7bjsNx5sJmAujEWKAOkQDD0CeQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['rados/test_health_warnings.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-21_10:47:12', 'tube': 'trial', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T11:56:30.108 DEBUG:teuthology.orchestra.run.trial196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T11:56:30.144 DEBUG:teuthology.orchestra.run.trial196:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T11:56:30.196 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:30.196 DEBUG:teuthology.orchestra.run.trial196:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T11:56:30.243 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T11:56:30.243 DEBUG:teuthology.orchestra.run.trial196:> stat /dev/vg_nvme/lv_1 2026-02-21T11:56:30.291 INFO:teuthology.orchestra.run.trial196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T11:56:30.291 INFO:teuthology.orchestra.run.trial196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T11:56:30.291 INFO:teuthology.orchestra.run.trial196.stdout:Device: 5h/5d Inode: 899 Links: 1 2026-02-21T11:56:30.291 INFO:teuthology.orchestra.run.trial196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T11:56:30.291 INFO:teuthology.orchestra.run.trial196.stdout:Access: 2026-02-21 11:55:36.523861082 +0000 2026-02-21T11:56:30.292 INFO:teuthology.orchestra.run.trial196.stdout:Modify: 2026-02-21 11:55:36.355863939 +0000 2026-02-21T11:56:30.292 INFO:teuthology.orchestra.run.trial196.stdout:Change: 2026-02-21 11:55:36.355863939 +0000 2026-02-21T11:56:30.292 INFO:teuthology.orchestra.run.trial196.stdout: Birth: - 2026-02-21T11:56:30.292 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T11:56:30.338 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records in 2026-02-21T11:56:30.338 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records out 2026-02-21T11:56:30.338 INFO:teuthology.orchestra.run.trial196.stderr:512 bytes copied, 0.000186376 s, 2.7 MB/s 2026-02-21T11:56:30.339 DEBUG:teuthology.orchestra.run.trial196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T11:56:30.385 DEBUG:teuthology.orchestra.run.trial196:> stat /dev/vg_nvme/lv_2 2026-02-21T11:56:30.431 INFO:teuthology.orchestra.run.trial196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout:Device: 5h/5d Inode: 920 Links: 1 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout:Access: 2026-02-21 11:55:36.863855303 +0000 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout:Modify: 2026-02-21 11:55:36.711857885 +0000 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout:Change: 2026-02-21 11:55:36.711857885 +0000 2026-02-21T11:56:30.432 INFO:teuthology.orchestra.run.trial196.stdout: Birth: - 2026-02-21T11:56:30.433 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T11:56:30.479 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records in 2026-02-21T11:56:30.479 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records out 2026-02-21T11:56:30.479 INFO:teuthology.orchestra.run.trial196.stderr:512 bytes copied, 0.000189552 s, 2.7 MB/s 2026-02-21T11:56:30.480 DEBUG:teuthology.orchestra.run.trial196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T11:56:30.529 DEBUG:teuthology.orchestra.run.trial196:> stat /dev/vg_nvme/lv_3 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout:Device: 5h/5d Inode: 944 Links: 1 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout:Access: 2026-02-21 11:55:37.235848976 +0000 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout:Modify: 2026-02-21 11:55:37.039852310 +0000 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout:Change: 2026-02-21 11:55:37.039852310 +0000 2026-02-21T11:56:30.580 INFO:teuthology.orchestra.run.trial196.stdout: Birth: - 2026-02-21T11:56:30.581 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T11:56:30.631 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records in 2026-02-21T11:56:30.631 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records out 2026-02-21T11:56:30.631 INFO:teuthology.orchestra.run.trial196.stderr:512 bytes copied, 0.0001938 s, 2.6 MB/s 2026-02-21T11:56:30.632 DEBUG:teuthology.orchestra.run.trial196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T11:56:30.685 DEBUG:teuthology.orchestra.run.trial196:> stat /dev/vg_nvme/lv_4 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout:Device: 5h/5d Inode: 969 Links: 1 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout:Access: 2026-02-21 11:55:37.571843264 +0000 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout:Modify: 2026-02-21 11:55:37.411845984 +0000 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout:Change: 2026-02-21 11:55:37.411845984 +0000 2026-02-21T11:56:30.735 INFO:teuthology.orchestra.run.trial196.stdout: Birth: - 2026-02-21T11:56:30.735 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T11:56:30.784 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records in 2026-02-21T11:56:30.784 INFO:teuthology.orchestra.run.trial196.stderr:1+0 records out 2026-02-21T11:56:30.784 INFO:teuthology.orchestra.run.trial196.stderr:512 bytes copied, 0.000169515 s, 3.0 MB/s 2026-02-21T11:56:30.784 DEBUG:teuthology.orchestra.run.trial196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T11:56:30.832 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:30.833 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial196.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-21T11:56:30.833 INFO:tasks.ceph:Generating config... 2026-02-21T11:56:30.833 INFO:tasks.ceph:[osd] osd max object name len = 400 2026-02-21T11:56:30.833 INFO:tasks.ceph:[osd] osd max object namespace len = 64 2026-02-21T11:56:30.833 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-21T11:56:30.833 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-21T11:56:30.834 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-21T11:56:30.834 INFO:tasks.ceph:[global] mon election default strategy = 3 2026-02-21T11:56:30.834 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T11:56:30.834 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T11:56:30.834 INFO:tasks.ceph:[mon] debug mon = 30 2026-02-21T11:56:30.834 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-21T11:56:30.834 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-21T11:56:30.834 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-21T11:56:30.834 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T11:56:30.834 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T11:56:30.892 INFO:teuthology.orchestra.run.trial196.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T11:56:30.895 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T11:56:30.959 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T11:56:31.012 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.196')] 2026-02-21T11:56:31.012 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.196', 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 30, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-21T11:56:31.014 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:31.014 DEBUG:teuthology.orchestra.run.trial196:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T11:56:31.055 DEBUG:teuthology.orchestra.run.trial196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.196 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:monmaptool: generated fsid 9c5344fb-5d4c-42a6-924b-048ca08c7e50 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:setting min_mon_release = tentacle 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:epoch 0 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:fsid 9c5344fb-5d4c-42a6-924b-048ca08c7e50 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:last_changed 2026-02-21T11:56:31.108736+0000 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:created 2026-02-21T11:56:31.108736+0000 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:min_mon_release 20 (tentacle) 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:election_strategy: 3 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:0: [v2:10.20.193.196:3300/0,v1:10.20.193.196:6789/0] mon.a 2026-02-21T11:56:31.112 INFO:teuthology.orchestra.run.trial196.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-21T11:56:31.113 DEBUG:teuthology.orchestra.run.trial196:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T11:56:31.160 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9c5344fb-5d4c-42a6-924b-048ca08c7e50... 2026-02-21T11:56:31.161 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T11:56:31.227 INFO:teuthology.orchestra.run.trial196.stdout:[global] 2026-02-21T11:56:31.227 INFO:teuthology.orchestra.run.trial196.stdout: chdir = "" 2026-02-21T11:56:31.227 INFO:teuthology.orchestra.run.trial196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T11:56:31.227 INFO:teuthology.orchestra.run.trial196.stdout: auth supported = cephx 2026-02-21T11:56:31.227 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.227 INFO:teuthology.orchestra.run.trial196.stdout: filestore xattr use omap = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon clock drift allowed = 1.000 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: osd crush chooseleaf type = 0 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: auth debug = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: ms die on old message = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: ms die on bug = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon pg warn max object skew = 0 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: osd pool default size = 2 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon osd allow primary affinity = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon osd allow pg remap = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon warn on legacy crush tunables = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon warn on crush straw calc version zero = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon warn on no sortbitwise = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon warn on osd down out interval zero = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon warn on too few osds = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon_allow_pool_size_one = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: osd default data pool replay window = 5 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon allow pool delete = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon cluster log file level = debug 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: debug asserts on shutdown = true 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon health detail to clog = false 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon host = 10.20.193.196 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: mon election default strategy = 3 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: fsid = 9c5344fb-5d4c-42a6-924b-048ca08c7e50 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.228 INFO:teuthology.orchestra.run.trial196.stdout:[osd] 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd journal size = 100 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd scrub load threshold = 5.0 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd scrub max interval = 600 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd mclock profile = high_recovery_ops 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd mclock skip benchmark = True 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd recover clone overlap = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd recovery max chunk = 1048576 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug shutdown = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug op order = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug verify stray on activate = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug trim objects = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd open classes on start = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug pg log writeout = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd deep scrub update digest min age = 30 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd map max advance = 10 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: journal zero on create = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: filestore ondisk finisher threads = 3 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: filestore apply finisher threads = 3 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: bdev debug aio = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug misdirected ops = true 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd max object name len = 400 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd max object namespace len = 64 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: bluestore zero block detection = True 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: debug ms = 1 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: debug osd = 20 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug verify cached snaps = True 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd debug verify missing on start = True 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd mclock override recovery settings = True 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd op queue = debug_random 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: osd op queue cut off = debug_random 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout:[mgr] 2026-02-21T11:56:31.229 INFO:teuthology.orchestra.run.trial196.stdout: debug ms = 1 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug mgr = 20 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug mon = 20 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug auth = 20 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon reweight min pgs per osd = 4 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon reweight min bytes per osd = 10 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mgr/telemetry/nag = false 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout:[mon] 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug ms = 1 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug mon = 30 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug paxos = 20 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: debug auth = 20 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon data avail warn = 5 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon mgr mkfs grace = 240 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon reweight min pgs per osd = 4 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon osd reporter subtree level = osd 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon osd prime pg temp = true 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon reweight min bytes per osd = 10 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: # 1m isn't quite enough 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon_down_mkfs_grace = 2m 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon_warn_on_filestore_osds = false 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: mon scrub interval = 300 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout:[client] 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: rgw cache enabled = true 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: rgw enable ops log = true 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: rgw enable usage log = true 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T11:56:31.230 INFO:teuthology.orchestra.run.trial196.stdout:[mon.a] 2026-02-21T11:56:31.236 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T11:56:31.237 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T11:56:31.306 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T11:56:31.306 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:31.306 DEBUG:teuthology.orchestra.run.trial196:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T11:56:31.356 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:31.356 DEBUG:teuthology.orchestra.run.trial196:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T11:56:31.404 INFO:tasks.ceph:Sending monmap to node ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:56:31.404 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:31.404 DEBUG:teuthology.orchestra.run.trial196:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T11:56:31.404 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T11:56:31.465 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:31.465 DEBUG:teuthology.orchestra.run.trial196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:31.515 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T11:56:31.515 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T11:56:31.516 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T11:56:31.583 INFO:teuthology.orchestra.run.trial196.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T11:56:31.586 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T11:56:31.586 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T11:56:31.586 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T11:56:31.645 INFO:teuthology.orchestra.run.trial196.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T11:56:31.652 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T11:56:31.652 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial196.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-21T11:56:31.652 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T11:56:31.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:31.703 INFO:tasks.ceph:role: osd.0 2026-02-21T11:56:31.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:56:31.703 DEBUG:teuthology.orchestra.run.trial196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout: = sunit=0 swidth=0 blks 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T11:56:31.753 INFO:teuthology.orchestra.run.trial196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T11:56:32.328 INFO:teuthology.orchestra.run.trial196.stdout:Discarding blocks...Done. 2026-02-21T11:56:32.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial196.front.sepia.ceph.com -o noatime 2026-02-21T11:56:32.330 DEBUG:teuthology.orchestra.run.trial196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T11:56:32.388 DEBUG:teuthology.orchestra.run.trial196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T11:56:32.437 INFO:teuthology.orchestra.run.trial196.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T11:56:32.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:56:32.438 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T11:56:32.487 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:32.487 INFO:tasks.ceph:role: osd.1 2026-02-21T11:56:32.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:56:32.487 DEBUG:teuthology.orchestra.run.trial196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T11:56:32.541 INFO:teuthology.orchestra.run.trial196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout: = sunit=0 swidth=0 blks 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T11:56:32.542 INFO:teuthology.orchestra.run.trial196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T11:56:33.107 INFO:teuthology.orchestra.run.trial196.stdout:Discarding blocks...Done. 2026-02-21T11:56:33.109 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial196.front.sepia.ceph.com -o noatime 2026-02-21T11:56:33.109 DEBUG:teuthology.orchestra.run.trial196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T11:56:33.169 DEBUG:teuthology.orchestra.run.trial196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T11:56:33.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:56:33.218 INFO:teuthology.orchestra.run.trial196.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T11:56:33.219 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T11:56:33.268 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:33.268 INFO:tasks.ceph:role: osd.2 2026-02-21T11:56:33.268 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:56:33.269 DEBUG:teuthology.orchestra.run.trial196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout: = sunit=0 swidth=0 blks 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T11:56:33.322 INFO:teuthology.orchestra.run.trial196.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T11:56:33.323 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T11:56:33.323 INFO:teuthology.orchestra.run.trial196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T11:56:33.883 INFO:teuthology.orchestra.run.trial196.stdout:Discarding blocks...Done. 2026-02-21T11:56:33.885 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial196.front.sepia.ceph.com -o noatime 2026-02-21T11:56:33.885 DEBUG:teuthology.orchestra.run.trial196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T11:56:33.940 DEBUG:teuthology.orchestra.run.trial196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T11:56:33.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:56:33.990 INFO:teuthology.orchestra.run.trial196.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T11:56:33.991 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-21T11:56:34.044 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:34.044 INFO:tasks.ceph:role: osd.3 2026-02-21T11:56:34.044 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:56:34.044 DEBUG:teuthology.orchestra.run.trial196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout: = sunit=0 swidth=0 blks 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T11:56:34.097 INFO:teuthology.orchestra.run.trial196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T11:56:34.658 INFO:teuthology.orchestra.run.trial196.stdout:Discarding blocks...Done. 2026-02-21T11:56:34.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial196.front.sepia.ceph.com -o noatime 2026-02-21T11:56:34.660 DEBUG:teuthology.orchestra.run.trial196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2026-02-21T11:56:34.720 DEBUG:teuthology.orchestra.run.trial196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-21T11:56:34.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:56:34.770 INFO:teuthology.orchestra.run.trial196.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T11:56:34.770 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-21T11:56:34.820 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:34.820 INFO:tasks.ceph:role: osd.4 2026-02-21T11:56:34.820 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-21T11:56:34.871 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:34.871 INFO:tasks.ceph:role: osd.5 2026-02-21T11:56:34.871 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-21T11:56:34.920 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:34.920 INFO:tasks.ceph:role: osd.6 2026-02-21T11:56:34.920 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-21T11:56:34.976 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:34.976 INFO:tasks.ceph:role: osd.7 2026-02-21T11:56:34.976 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2026-02-21T11:56:35.023 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:35.023 INFO:tasks.ceph:role: osd.8 2026-02-21T11:56:35.023 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2026-02-21T11:56:35.072 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T11:56:35.072 INFO:tasks.ceph:role: osd.9 2026-02-21T11:56:35.072 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:35.137 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.129+0000 7f80bd49aa40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-21T11:56:35.138 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.129+0000 7f80bd49aa40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-21T11:56:35.138 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.129+0000 7f80bd49aa40 -1 bdev(0x55799783b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:35.138 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.129+0000 7f80bd49aa40 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-21T11:56:35.805 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T11:56:35.857 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:35.925 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.917+0000 7f06f943aa40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-21T11:56:35.926 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.917+0000 7f06f943aa40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-21T11:56:35.926 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.917+0000 7f06f943aa40 -1 bdev(0x563967b29800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:35.927 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:35.917+0000 7f06f943aa40 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-21T11:56:36.588 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T11:56:36.644 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:36.708 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:36.701+0000 7efd046c7a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-21T11:56:36.709 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:36.701+0000 7efd046c7a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-21T11:56:36.709 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:36.701+0000 7efd046c7a40 -1 bdev(0x55a4e65fd800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:36.709 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:36.701+0000 7efd046c7a40 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-21T11:56:37.353 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T11:56:37.404 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:37.469 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:37.461+0000 7fbe98a7ca40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-21T11:56:37.470 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:37.461+0000 7fbe98a7ca40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-21T11:56:37.470 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:37.461+0000 7fbe98a7ca40 -1 bdev(0x560f474ed800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:37.470 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:37.461+0000 7fbe98a7ca40 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-21T11:56:38.143 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-21T11:56:38.196 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:38.261 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:38.249+0000 7f04dbffca40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-21T11:56:38.261 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:38.249+0000 7f04dbffca40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-21T11:56:38.261 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:38.253+0000 7f04dbffca40 -1 bdev(0x55ac598ad800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:38.261 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:38.253+0000 7f04dbffca40 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-21T11:56:38.936 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-21T11:56:38.989 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:39.053 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.045+0000 7f557ab11a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-21T11:56:39.054 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.045+0000 7f557ab11a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-21T11:56:39.054 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.045+0000 7f557ab11a40 -1 bdev(0x55e96efaf800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:39.054 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.045+0000 7f557ab11a40 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-21T11:56:39.616 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-21T11:56:39.670 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:39.733 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.725+0000 7fa08f535a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-21T11:56:39.733 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.725+0000 7fa08f535a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-21T11:56:39.733 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.725+0000 7fa08f535a40 -1 bdev(0x557798671800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:39.733 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:39.725+0000 7fa08f535a40 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-21T11:56:40.472 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-21T11:56:40.525 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:40.589 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:40.581+0000 7f0eedb45a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-21T11:56:40.589 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:40.581+0000 7f0eedb45a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-21T11:56:40.589 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:40.581+0000 7f0eedb45a40 -1 bdev(0x55ed92c4b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:40.589 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:40.581+0000 7f0eedb45a40 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-21T11:56:41.228 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-21T11:56:41.282 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:41.345 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:41.337+0000 7f3a08ebca40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2026-02-21T11:56:41.345 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:41.337+0000 7f3a08ebca40 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2026-02-21T11:56:41.345 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:41.337+0000 7f3a08ebca40 -1 bdev(0x559c1fac3800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:41.345 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:41.337+0000 7f3a08ebca40 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2026-02-21T11:56:41.956 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2026-02-21T11:56:42.009 DEBUG:teuthology.orchestra.run.trial196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:42.073 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:42.065+0000 7fda2c081a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2026-02-21T11:56:42.073 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:42.065+0000 7fda2c081a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2026-02-21T11:56:42.073 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:42.065+0000 7fda2c081a40 -1 bdev(0x5578efc7d800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2026-02-21T11:56:42.073 INFO:teuthology.orchestra.run.trial196.stderr:2026-02-21T11:56:42.065+0000 7fda2c081a40 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2026-02-21T11:56:42.729 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2026-02-21T11:56:42.781 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T11:56:42.781 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:42.781 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T11:56:42.837 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:42.837 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T11:56:42.893 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:42.893 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T11:56:42.947 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:42.947 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T11:56:42.997 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:42.997 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-21T11:56:43.047 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.047 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-21T11:56:43.101 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.101 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-21T11:56:43.152 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.152 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-21T11:56:43.204 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.204 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-21T11:56:43.255 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.255 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2026-02-21T11:56:43.305 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.305 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2026-02-21T11:56:43.357 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:43.357 DEBUG:teuthology.orchestra.run.trial196:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T11:56:43.403 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T11:56:43.404 DEBUG:teuthology.orchestra.run.trial196:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[mgr.x] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQBvnZlpsLuWIhAANmDRLSNAKKMuV1lTJd4Kww== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.0] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQBznZlphpv2BxAA0ZOsk0Rs5uedE7vKEZ3icg== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.1] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQBznZlp5Jr1NhAAQW8U4MYX4qaOOgTtksUq6w== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.2] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB0nZlpkCQDKhAAo42YO0t8+wn4m/JY5vygaw== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.3] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB1nZlp5P66GxAAqwQ8zYbV1z0rdUmVWlLN6w== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.4] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB2nZlp05xIDxAAioDXK4ucRCXnmSOAVP9V0A== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.5] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB3nZlp1OT5AhAAMnPiySDR/QkjpxATgd7OMA== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.6] 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB3nZlpteZ6KxAA9rvZIw0RDYyMF0YUjMcpkg== 2026-02-21T11:56:43.451 INFO:teuthology.orchestra.run.trial196.stdout:[osd.7] 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB4nZlpO8jcIhAA+Ato+n/abmOM9ueRt53o/w== 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout:[osd.8] 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB5nZlpPgZXFBAAq2TZS6n2Uoktnr7lxpBjFQ== 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout:[osd.9] 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout: key = AQB6nZlpUAMYBBAA6iBWB0bwp5kpMjxW0J5PkQ== 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout:[client.0] 2026-02-21T11:56:43.452 INFO:teuthology.orchestra.run.trial196.stdout: key = AQBvnZlpXes+JhAA0G9QmleH1ncjP3QcwlPMng== 2026-02-21T11:56:43.452 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T11:56:43.522 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.586 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.649 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.713 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.776 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.841 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.905 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:43.971 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:44.033 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:44.096 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T11:56:44.161 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T11:56:44.226 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T11:56:44.227 DEBUG:teuthology.orchestra.run.trial196:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T11:56:44.276 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T11:56:44.345 DEBUG:teuthology.orchestra.run.trial196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T11:56:44.400 DEBUG:teuthology.orchestra.run.trial196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T11:56:44.447 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T11:56:44.447 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T11:56:44.447 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T11:56:44.489 INFO:tasks.ceph.mon.a:Started 2026-02-21T11:56:44.489 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph config set mgr mgr_pool false --force 2026-02-21T11:56:44.626 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T11:56:44.626 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T11:56:44.626 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T11:56:44.627 INFO:tasks.ceph.mgr.x:Started 2026-02-21T11:56:44.628 DEBUG:tasks.ceph:set 0 configs 2026-02-21T11:56:44.628 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph config dump 2026-02-21T11:56:44.723 INFO:teuthology.orchestra.run.trial196.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T11:56:44.723 INFO:teuthology.orchestra.run.trial196.stdout:mgr dev mgr_pool false * 2026-02-21T11:56:44.735 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T11:56:44.735 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T11:56:44.836 INFO:teuthology.orchestra.run.trial196.stderr:adjusted tunables profile to default 2026-02-21T11:56:44.846 INFO:tasks.ceph:check_enable_crimson: False 2026-02-21T11:56:44.846 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T11:56:44.846 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:44.846 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T11:56:44.855 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:44.855 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T11:56:44.904 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:44.904 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T11:56:44.950 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:44.950 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-21T11:56:44.998 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:44.998 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-21T11:56:45.047 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:45.047 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-21T11:56:45.094 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:45.094 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-21T11:56:45.142 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:45.142 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-21T11:56:45.191 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:45.191 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2026-02-21T11:56:45.241 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:56:45.241 DEBUG:teuthology.orchestra.run.trial196:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2026-02-21T11:56:45.291 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new 23153a6f-a5f3-44a2-aacd-3d049090cd7c 0 2026-02-21T11:56:45.429 INFO:teuthology.orchestra.run.trial196.stdout:0 2026-02-21T11:56:45.438 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new a3359d61-4602-4a4e-8224-f7bd08750521 1 2026-02-21T11:56:45.535 INFO:teuthology.orchestra.run.trial196.stdout:1 2026-02-21T11:56:45.544 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new 3a9f2159-17a6-4b49-bf3c-1ada498cfa39 2 2026-02-21T11:56:45.640 INFO:teuthology.orchestra.run.trial196.stdout:2 2026-02-21T11:56:45.650 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new 5533f509-0f80-4f4f-b3d1-7ec1b80d7f64 3 2026-02-21T11:56:45.749 INFO:teuthology.orchestra.run.trial196.stdout:3 2026-02-21T11:56:45.759 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new ac04a66c-f8db-4ada-8bb7-974605771fee 4 2026-02-21T11:56:45.857 INFO:teuthology.orchestra.run.trial196.stdout:4 2026-02-21T11:56:45.866 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new b8f758d2-6548-44b8-ab83-55138eeb765e 5 2026-02-21T11:56:45.965 INFO:teuthology.orchestra.run.trial196.stdout:5 2026-02-21T11:56:45.974 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new 6f0ba2be-32a4-4a70-a952-2980d9bf9801 6 2026-02-21T11:56:46.073 INFO:teuthology.orchestra.run.trial196.stdout:6 2026-02-21T11:56:46.082 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new f3ae4755-f066-4372-ae0c-f364ea536d18 7 2026-02-21T11:56:46.182 INFO:teuthology.orchestra.run.trial196.stdout:7 2026-02-21T11:56:46.183 INFO:tasks.ceph.mgr.x.trial196.stderr:ERROR:root:Module 'jmespath' is not installed. 2026-02-21T11:56:46.192 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new 05cc8f5d-d979-4eb7-93c2-5484939706e6 8 2026-02-21T11:56:46.240 INFO:tasks.ceph.mgr.x.trial196.stderr:ERROR:dashboard.controllers.nvmeof:Failed to import NVMeoFClient and related components: cannot import name 'NVMeoFClient' from 'dashboard.services.nvmeof_client' (/usr/share/ceph/mgr/dashboard/services/nvmeof_client.py) 2026-02-21T11:56:46.245 INFO:tasks.ceph.mgr.x.trial196.stderr:ERROR:sso:Module 'jmespath' is not installed. 2026-02-21T11:56:46.291 INFO:teuthology.orchestra.run.trial196.stdout:8 2026-02-21T11:56:46.299 INFO:tasks.ceph.mgr.x.trial196.stderr:ERROR:root:Module 'xmltodict' is not installed. 2026-02-21T11:56:46.300 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd new e8713fc9-8526-41b8-91e8-573218e1b699 9 2026-02-21T11:56:46.399 INFO:teuthology.orchestra.run.trial196.stdout:9 2026-02-21T11:56:46.408 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T11:56:46.408 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-21T11:56:46.409 INFO:tasks.ceph.osd.0:Started 2026-02-21T11:56:46.409 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T11:56:46.409 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-21T11:56:46.410 INFO:tasks.ceph.osd.1:Started 2026-02-21T11:56:46.410 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T11:56:46.410 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-21T11:56:46.411 INFO:tasks.ceph.osd.2:Started 2026-02-21T11:56:46.411 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-21T11:56:46.411 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-21T11:56:46.412 INFO:tasks.ceph.osd.3:Started 2026-02-21T11:56:46.412 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-21T11:56:46.412 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2026-02-21T11:56:46.413 INFO:tasks.ceph.osd.4:Started 2026-02-21T11:56:46.413 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-21T11:56:46.413 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2026-02-21T11:56:46.414 INFO:tasks.ceph.osd.5:Started 2026-02-21T11:56:46.414 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-21T11:56:46.414 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2026-02-21T11:56:46.416 INFO:tasks.ceph.osd.6:Started 2026-02-21T11:56:46.416 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-21T11:56:46.416 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2026-02-21T11:56:46.417 INFO:tasks.ceph.osd.7:Started 2026-02-21T11:56:46.417 INFO:tasks.ceph.osd.8:Restarting daemon 2026-02-21T11:56:46.417 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2026-02-21T11:56:46.418 INFO:tasks.ceph.osd.8:Started 2026-02-21T11:56:46.418 INFO:tasks.ceph.osd.9:Restarting daemon 2026-02-21T11:56:46.418 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2026-02-21T11:56:46.418 INFO:tasks.ceph.osd.9:Started 2026-02-21T11:56:46.419 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T11:56:46.511 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:46.511 INFO:teuthology.orchestra.run.trial196.stdout:{"epoch":12,"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","created":"2026-02-21T11:56:44.569498+0000","modified":"2026-02-21T11:56:46.394335+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T11:56:46.394335+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"23153a6f-a5f3-44a2-aacd-3d049090cd7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a3359d61-4602-4a4e-8224-f7bd08750521","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3a9f2159-17a6-4b49-bf3c-1ada498cfa39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5533f509-0f80-4f4f-b3d1-7ec1b80d7f64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac04a66c-f8db-4ada-8bb7-974605771fee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b8f758d2-6548-44b8-ab83-55138eeb765e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f0ba2be-32a4-4a70-a952-2980d9bf9801","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3ae4755-f066-4372-ae0c-f364ea536d18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"05cc8f5d-d979-4eb7-93c2-5484939706e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"e8713fc9-8526-41b8-91e8-573218e1b699","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T11:56:46.520 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T11:56:46.520 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T11:56:46.754 INFO:tasks.ceph.mgr.x.trial196.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-21T11:56:46.754 INFO:tasks.ceph.mgr.x.trial196.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-21T11:56:46.754 INFO:tasks.ceph.mgr.x.trial196.stderr: from numpy import show_config as show_numpy_config 2026-02-21T11:56:46.921 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:56:46.913+0000 7fa9b5888a40 -1 Falling back to public interface 2026-02-21T11:56:46.929 INFO:tasks.ceph.osd.4.trial196.stderr:2026-02-21T11:56:46.921+0000 7f0f20d8aa40 -1 Falling back to public interface 2026-02-21T11:56:46.929 INFO:tasks.ceph.osd.6.trial196.stderr:2026-02-21T11:56:46.921+0000 7f71859dba40 -1 Falling back to public interface 2026-02-21T11:56:46.945 INFO:tasks.ceph.osd.3.trial196.stderr:2026-02-21T11:56:46.937+0000 7faa44363a40 -1 Falling back to public interface 2026-02-21T11:56:46.946 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:56:46.937+0000 7fbe32a3fa40 -1 Falling back to public interface 2026-02-21T11:56:46.946 INFO:tasks.ceph.osd.7.trial196.stderr:2026-02-21T11:56:46.937+0000 7f2310015a40 -1 Falling back to public interface 2026-02-21T11:56:46.949 INFO:tasks.ceph.osd.5.trial196.stderr:2026-02-21T11:56:46.941+0000 7f57ae782a40 -1 Falling back to public interface 2026-02-21T11:56:46.957 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:56:46.949+0000 7fddcb14ba40 -1 Falling back to public interface 2026-02-21T11:56:46.977 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:56:46.969+0000 7f5d7b935a40 -1 Falling back to public interface 2026-02-21T11:56:46.989 INFO:tasks.ceph.osd.2.trial196.stderr:2026-02-21T11:56:46.981+0000 7f611bc50a40 -1 Falling back to public interface 2026-02-21T11:56:47.525 DEBUG:teuthology.orchestra.run.trial196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T11:56:47.630 INFO:teuthology.misc.health.trial196.stdout: 2026-02-21T11:56:47.631 INFO:teuthology.misc.health.trial196.stdout:{"epoch":12,"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","created":"2026-02-21T11:56:44.569498+0000","modified":"2026-02-21T11:56:46.394335+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T11:56:46.394335+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"23153a6f-a5f3-44a2-aacd-3d049090cd7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a3359d61-4602-4a4e-8224-f7bd08750521","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3a9f2159-17a6-4b49-bf3c-1ada498cfa39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5533f509-0f80-4f4f-b3d1-7ec1b80d7f64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac04a66c-f8db-4ada-8bb7-974605771fee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b8f758d2-6548-44b8-ab83-55138eeb765e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f0ba2be-32a4-4a70-a952-2980d9bf9801","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3ae4755-f066-4372-ae0c-f364ea536d18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"05cc8f5d-d979-4eb7-93c2-5484939706e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"e8713fc9-8526-41b8-91e8-573218e1b699","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T11:56:47.641 DEBUG:teuthology.misc:0 of 10 OSDs are up 2026-02-21T11:56:47.750 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:56:47.741+0000 7fa9b5888a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:56:47.776 INFO:tasks.ceph.osd.4.trial196.stderr:2026-02-21T11:56:47.765+0000 7f0f20d8aa40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:56:47.812 INFO:tasks.ceph.osd.6.trial196.stderr:2026-02-21T11:56:47.801+0000 7f71859dba40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:56:47.828 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:56:47.817+0000 7f5d7b935a40 -1 osd.8 0 log_to_monitors true 2026-02-21T11:56:47.829 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:56:47.821+0000 7fbe32a3fa40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:56:47.830 INFO:tasks.ceph.mgr.x.trial196.stderr:2026-02-21T11:56:47.821+0000 7f38fc0b1640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-21T11:56:47.855 INFO:tasks.ceph.osd.3.trial196.stderr:2026-02-21T11:56:47.845+0000 7faa44363a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:56:47.859 INFO:tasks.ceph.osd.5.trial196.stderr:2026-02-21T11:56:47.849+0000 7f57ae782a40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:56:47.876 INFO:tasks.ceph.osd.7.trial196.stderr:2026-02-21T11:56:47.865+0000 7f2310015a40 -1 osd.7 0 log_to_monitors true 2026-02-21T11:56:47.896 INFO:tasks.ceph.osd.2.trial196.stderr:2026-02-21T11:56:47.885+0000 7f611bc50a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:56:47.956 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:56:47.945+0000 7fddcb14ba40 -1 osd.9 0 log_to_monitors true 2026-02-21T11:56:49.590 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:56:49.581+0000 7fa9b1832640 -1 osd.0 0 waiting for initial osdmap 2026-02-21T11:56:49.590 INFO:tasks.ceph.osd.4.trial196.stderr:2026-02-21T11:56:49.581+0000 7f0f1cd34640 -1 osd.4 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:56:49.581+0000 7f5d778df640 -1 osd.8 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.6.trial196.stderr:2026-02-21T11:56:49.581+0000 7f7181985640 -1 osd.6 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.3.trial196.stderr:2026-02-21T11:56:49.581+0000 7faa4030d640 -1 osd.3 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.5.trial196.stderr:2026-02-21T11:56:49.581+0000 7f57aa72c640 -1 osd.5 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:56:49.581+0000 7fbe2e9e9640 -1 osd.1 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.7.trial196.stderr:2026-02-21T11:56:49.581+0000 7f230bfbf640 -1 osd.7 0 waiting for initial osdmap 2026-02-21T11:56:49.591 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:56:49.581+0000 7fddc70f5640 -1 osd.9 0 waiting for initial osdmap 2026-02-21T11:56:49.592 INFO:tasks.ceph.osd.2.trial196.stderr:2026-02-21T11:56:49.581+0000 7f6117bfa640 -1 osd.2 0 waiting for initial osdmap 2026-02-21T11:56:49.593 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:56:49.585+0000 7fa9a8f01640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.594 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:56:49.585+0000 7f5d72fb6640 -1 osd.8 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.595 INFO:tasks.ceph.osd.4.trial196.stderr:2026-02-21T11:56:49.585+0000 7f0f14403640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.595 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:56:49.585+0000 7fbe260b8640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.595 INFO:tasks.ceph.osd.3.trial196.stderr:2026-02-21T11:56:49.585+0000 7faa379dc640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.595 INFO:tasks.ceph.osd.5.trial196.stderr:2026-02-21T11:56:49.585+0000 7f57a5e03640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.595 INFO:tasks.ceph.osd.2.trial196.stderr:2026-02-21T11:56:49.585+0000 7f61132d1640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.596 INFO:tasks.ceph.osd.6.trial196.stderr:2026-02-21T11:56:49.585+0000 7f7179054640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.596 INFO:tasks.ceph.osd.7.trial196.stderr:2026-02-21T11:56:49.585+0000 7f2307696640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:49.596 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:56:49.585+0000 7fddbe7c4640 -1 osd.9 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:56:54.647 DEBUG:teuthology.orchestra.run.trial196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T11:56:54.799 INFO:teuthology.misc.health.trial196.stdout: 2026-02-21T11:56:54.800 INFO:teuthology.misc.health.trial196.stdout:{"epoch":15,"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","created":"2026-02-21T11:56:44.569498+0000","modified":"2026-02-21T11:56:50.582286+0000","last_up_change":"2026-02-21T11:56:50.582286+0000","last_in_change":"2026-02-21T11:56:46.394335+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"23153a6f-a5f3-44a2-aacd-3d049090cd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6802","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6803","nonce":1308491399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6804","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6805","nonce":1308491399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6808","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6809","nonce":1308491399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6806","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6807","nonce":1308491399}]},"public_addr":"10.20.193.196:6803/1308491399","cluster_addr":"10.20.193.196:6805/1308491399","heartbeat_back_addr":"10.20.193.196:6809/1308491399","heartbeat_front_addr":"10.20.193.196:6807/1308491399","state":["exists","up"]},{"osd":1,"uuid":"a3359d61-4602-4a4e-8224-f7bd08750521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6829","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6833","nonce":2003661087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6836","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6840","nonce":2003661087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6851","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6855","nonce":2003661087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6843","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6847","nonce":2003661087}]},"public_addr":"10.20.193.196:6833/2003661087","cluster_addr":"10.20.193.196:6840/2003661087","heartbeat_back_addr":"10.20.193.196:6855/2003661087","heartbeat_front_addr":"10.20.193.196:6847/2003661087","state":["exists","up"]},{"osd":2,"uuid":"3a9f2159-17a6-4b49-bf3c-1ada498cfa39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6874","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6875","nonce":1372045057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6876","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6877","nonce":1372045057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6880","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6881","nonce":1372045057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6878","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6879","nonce":1372045057}]},"public_addr":"10.20.193.196:6875/1372045057","cluster_addr":"10.20.193.196:6877/1372045057","heartbeat_back_addr":"10.20.193.196:6881/1372045057","heartbeat_front_addr":"10.20.193.196:6879/1372045057","state":["exists","up"]},{"osd":3,"uuid":"5533f509-0f80-4f4f-b3d1-7ec1b80d7f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6826","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6827","nonce":635434592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6830","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6832","nonce":635434592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6838","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6841","nonce":635434592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6834","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6837","nonce":635434592}]},"public_addr":"10.20.193.196:6827/635434592","cluster_addr":"10.20.193.196:6832/635434592","heartbeat_back_addr":"10.20.193.196:6841/635434592","heartbeat_front_addr":"10.20.193.196:6837/635434592","state":["exists","up"]},{"osd":4,"uuid":"ac04a66c-f8db-4ada-8bb7-974605771fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6810","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6812","nonce":627255440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6814","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6816","nonce":627255440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6821","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6823","nonce":627255440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6817","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6819","nonce":627255440}]},"public_addr":"10.20.193.196:6812/627255440","cluster_addr":"10.20.193.196:6816/627255440","heartbeat_back_addr":"10.20.193.196:6823/627255440","heartbeat_front_addr":"10.20.193.196:6819/627255440","state":["exists","up"]},{"osd":5,"uuid":"b8f758d2-6548-44b8-ab83-55138eeb765e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6844","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6846","nonce":1440695253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6848","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6850","nonce":1440695253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6856","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6857","nonce":1440695253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6852","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6854","nonce":1440695253}]},"public_addr":"10.20.193.196:6846/1440695253","cluster_addr":"10.20.193.196:6850/1440695253","heartbeat_back_addr":"10.20.193.196:6857/1440695253","heartbeat_front_addr":"10.20.193.196:6854/1440695253","state":["exists","up"]},{"osd":6,"uuid":"6f0ba2be-32a4-4a70-a952-2980d9bf9801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6811","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6813","nonce":2180386325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6815","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6818","nonce":2180386325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6824","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6825","nonce":2180386325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6820","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6822","nonce":2180386325}]},"public_addr":"10.20.193.196:6813/2180386325","cluster_addr":"10.20.193.196:6818/2180386325","heartbeat_back_addr":"10.20.193.196:6825/2180386325","heartbeat_front_addr":"10.20.193.196:6822/2180386325","state":["exists","up"]},{"osd":7,"uuid":"f3ae4755-f066-4372-ae0c-f364ea536d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6828","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6831","nonce":1122662840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6835","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6839","nonce":1122662840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6849","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6853","nonce":1122662840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6842","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6845","nonce":1122662840}]},"public_addr":"10.20.193.196:6831/1122662840","cluster_addr":"10.20.193.196:6839/1122662840","heartbeat_back_addr":"10.20.193.196:6853/1122662840","heartbeat_front_addr":"10.20.193.196:6845/1122662840","state":["exists","up"]},{"osd":8,"uuid":"05cc8f5d-d979-4eb7-93c2-5484939706e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6866","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6867","nonce":2631587950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6868","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6869","nonce":2631587950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6872","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6873","nonce":2631587950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6870","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6871","nonce":2631587950}]},"public_addr":"10.20.193.196:6867/2631587950","cluster_addr":"10.20.193.196:6869/2631587950","heartbeat_back_addr":"10.20.193.196:6873/2631587950","heartbeat_front_addr":"10.20.193.196:6871/2631587950","state":["exists","up"]},{"osd":9,"uuid":"e8713fc9-8526-41b8-91e8-573218e1b699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6858","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6859","nonce":1536643720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6860","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6861","nonce":1536643720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6864","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6865","nonce":1536643720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6862","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6863","nonce":1536643720}]},"public_addr":"10.20.193.196:6859/1536643720","cluster_addr":"10.20.193.196:6861/1536643720","heartbeat_back_addr":"10.20.193.196:6865/1536643720","heartbeat_front_addr":"10.20.193.196:6863/1536643720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T11:56:54.810 DEBUG:teuthology.misc:10 of 10 OSDs are up 2026-02-21T11:56:54.810 INFO:tasks.ceph:Creating RBD pool 2026-02-21T11:56:54.810 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T11:56:55.597 INFO:teuthology.orchestra.run.trial196.stderr:pool 'rbd' created 2026-02-21T11:56:55.609 DEBUG:teuthology.orchestra.run.trial196:> rbd --cluster ceph pool init rbd 2026-02-21T11:56:58.612 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T11:56:58.613 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T11:56:58.613 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T11:56:58.773 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:58.783 INFO:teuthology.orchestra.run.trial196.stdout:[{"version":2,"timestamp":"2026-02-21T11:56:44.609978+0000","name":"","changes":[{"name":"mgr/mgr_pool","new_value":"false"}]}] 2026-02-21T11:56:58.783 INFO:tasks.ceph_manager:config epoch is 2 2026-02-21T11:56:58.783 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T11:56:58.783 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T11:56:58.783 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T11:56:58.961 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:58.971 INFO:teuthology.orchestra.run.trial196.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6800","nonce":257410844},{"type":"v1","addr":"10.20.193.196:6801","nonce":257410844}]},"active_addr":"10.20.193.196:6801/257410844","active_change":"2026-02-21T11:56:46.821282+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.196:0","nonce":2634509293}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.196:0","nonce":3225748107}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.196:0","nonce":794954102}]}]} 2026-02-21T11:56:58.973 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T11:56:58.973 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T11:56:58.973 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T11:56:59.136 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:59.136 INFO:teuthology.orchestra.run.trial196.stdout:{"epoch":19,"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","created":"2026-02-21T11:56:44.569498+0000","modified":"2026-02-21T11:56:58.597263+0000","last_up_change":"2026-02-21T11:56:50.582286+0000","last_in_change":"2026-02-21T11:56:46.394335+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T11:56:54.964058+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.89999997615814209,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1.0000001192092896,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23153a6f-a5f3-44a2-aacd-3d049090cd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6802","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6803","nonce":1308491399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6804","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6805","nonce":1308491399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6808","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6809","nonce":1308491399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6806","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6807","nonce":1308491399}]},"public_addr":"10.20.193.196:6803/1308491399","cluster_addr":"10.20.193.196:6805/1308491399","heartbeat_back_addr":"10.20.193.196:6809/1308491399","heartbeat_front_addr":"10.20.193.196:6807/1308491399","state":["exists","up"]},{"osd":1,"uuid":"a3359d61-4602-4a4e-8224-f7bd08750521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6829","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6833","nonce":2003661087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6836","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6840","nonce":2003661087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6851","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6855","nonce":2003661087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6843","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6847","nonce":2003661087}]},"public_addr":"10.20.193.196:6833/2003661087","cluster_addr":"10.20.193.196:6840/2003661087","heartbeat_back_addr":"10.20.193.196:6855/2003661087","heartbeat_front_addr":"10.20.193.196:6847/2003661087","state":["exists","up"]},{"osd":2,"uuid":"3a9f2159-17a6-4b49-bf3c-1ada498cfa39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6874","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6875","nonce":1372045057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6876","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6877","nonce":1372045057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6880","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6881","nonce":1372045057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6878","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6879","nonce":1372045057}]},"public_addr":"10.20.193.196:6875/1372045057","cluster_addr":"10.20.193.196:6877/1372045057","heartbeat_back_addr":"10.20.193.196:6881/1372045057","heartbeat_front_addr":"10.20.193.196:6879/1372045057","state":["exists","up"]},{"osd":3,"uuid":"5533f509-0f80-4f4f-b3d1-7ec1b80d7f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6826","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6827","nonce":635434592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6830","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6832","nonce":635434592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6838","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6841","nonce":635434592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6834","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6837","nonce":635434592}]},"public_addr":"10.20.193.196:6827/635434592","cluster_addr":"10.20.193.196:6832/635434592","heartbeat_back_addr":"10.20.193.196:6841/635434592","heartbeat_front_addr":"10.20.193.196:6837/635434592","state":["exists","up"]},{"osd":4,"uuid":"ac04a66c-f8db-4ada-8bb7-974605771fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6810","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6812","nonce":627255440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6814","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6816","nonce":627255440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6821","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6823","nonce":627255440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6817","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6819","nonce":627255440}]},"public_addr":"10.20.193.196:6812/627255440","cluster_addr":"10.20.193.196:6816/627255440","heartbeat_back_addr":"10.20.193.196:6823/627255440","heartbeat_front_addr":"10.20.193.196:6819/627255440","state":["exists","up"]},{"osd":5,"uuid":"b8f758d2-6548-44b8-ab83-55138eeb765e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6844","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6846","nonce":1440695253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6848","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6850","nonce":1440695253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6856","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6857","nonce":1440695253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6852","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6854","nonce":1440695253}]},"public_addr":"10.20.193.196:6846/1440695253","cluster_addr":"10.20.193.196:6850/1440695253","heartbeat_back_addr":"10.20.193.196:6857/1440695253","heartbeat_front_addr":"10.20.193.196:6854/1440695253","state":["exists","up"]},{"osd":6,"uuid":"6f0ba2be-32a4-4a70-a952-2980d9bf9801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6811","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6813","nonce":2180386325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6815","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6818","nonce":2180386325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6824","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6825","nonce":2180386325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6820","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6822","nonce":2180386325}]},"public_addr":"10.20.193.196:6813/2180386325","cluster_addr":"10.20.193.196:6818/2180386325","heartbeat_back_addr":"10.20.193.196:6825/2180386325","heartbeat_front_addr":"10.20.193.196:6822/2180386325","state":["exists","up"]},{"osd":7,"uuid":"f3ae4755-f066-4372-ae0c-f364ea536d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6828","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6831","nonce":1122662840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6835","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6839","nonce":1122662840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6849","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6853","nonce":1122662840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6842","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6845","nonce":1122662840}]},"public_addr":"10.20.193.196:6831/1122662840","cluster_addr":"10.20.193.196:6839/1122662840","heartbeat_back_addr":"10.20.193.196:6853/1122662840","heartbeat_front_addr":"10.20.193.196:6845/1122662840","state":["exists","up"]},{"osd":8,"uuid":"05cc8f5d-d979-4eb7-93c2-5484939706e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6866","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6867","nonce":2631587950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6868","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6869","nonce":2631587950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6872","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6873","nonce":2631587950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6870","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6871","nonce":2631587950}]},"public_addr":"10.20.193.196:6867/2631587950","cluster_addr":"10.20.193.196:6869/2631587950","heartbeat_back_addr":"10.20.193.196:6873/2631587950","heartbeat_front_addr":"10.20.193.196:6871/2631587950","state":["exists","up"]},{"osd":9,"uuid":"e8713fc9-8526-41b8-91e8-573218e1b699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6858","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6859","nonce":1536643720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6860","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6861","nonce":1536643720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6864","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6865","nonce":1536643720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6862","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6863","nonce":1536643720}]},"public_addr":"10.20.193.196:6859/1536643720","cluster_addr":"10.20.193.196:6861/1536643720","heartbeat_back_addr":"10.20.193.196:6865/1536643720","heartbeat_front_addr":"10.20.193.196:6863/1536643720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T11:56:59.146 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T11:56:59.147 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T11:56:59.300 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:56:59.301 INFO:teuthology.orchestra.run.trial196.stdout:{"epoch":19,"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","created":"2026-02-21T11:56:44.569498+0000","modified":"2026-02-21T11:56:58.597263+0000","last_up_change":"2026-02-21T11:56:50.582286+0000","last_in_change":"2026-02-21T11:56:46.394335+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T11:56:54.964058+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.89999997615814209,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1.0000001192092896,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23153a6f-a5f3-44a2-aacd-3d049090cd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6802","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6803","nonce":1308491399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6804","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6805","nonce":1308491399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6808","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6809","nonce":1308491399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6806","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6807","nonce":1308491399}]},"public_addr":"10.20.193.196:6803/1308491399","cluster_addr":"10.20.193.196:6805/1308491399","heartbeat_back_addr":"10.20.193.196:6809/1308491399","heartbeat_front_addr":"10.20.193.196:6807/1308491399","state":["exists","up"]},{"osd":1,"uuid":"a3359d61-4602-4a4e-8224-f7bd08750521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6829","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6833","nonce":2003661087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6836","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6840","nonce":2003661087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6851","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6855","nonce":2003661087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6843","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6847","nonce":2003661087}]},"public_addr":"10.20.193.196:6833/2003661087","cluster_addr":"10.20.193.196:6840/2003661087","heartbeat_back_addr":"10.20.193.196:6855/2003661087","heartbeat_front_addr":"10.20.193.196:6847/2003661087","state":["exists","up"]},{"osd":2,"uuid":"3a9f2159-17a6-4b49-bf3c-1ada498cfa39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6874","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6875","nonce":1372045057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6876","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6877","nonce":1372045057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6880","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6881","nonce":1372045057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6878","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6879","nonce":1372045057}]},"public_addr":"10.20.193.196:6875/1372045057","cluster_addr":"10.20.193.196:6877/1372045057","heartbeat_back_addr":"10.20.193.196:6881/1372045057","heartbeat_front_addr":"10.20.193.196:6879/1372045057","state":["exists","up"]},{"osd":3,"uuid":"5533f509-0f80-4f4f-b3d1-7ec1b80d7f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6826","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6827","nonce":635434592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6830","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6832","nonce":635434592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6838","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6841","nonce":635434592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6834","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6837","nonce":635434592}]},"public_addr":"10.20.193.196:6827/635434592","cluster_addr":"10.20.193.196:6832/635434592","heartbeat_back_addr":"10.20.193.196:6841/635434592","heartbeat_front_addr":"10.20.193.196:6837/635434592","state":["exists","up"]},{"osd":4,"uuid":"ac04a66c-f8db-4ada-8bb7-974605771fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6810","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6812","nonce":627255440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6814","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6816","nonce":627255440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6821","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6823","nonce":627255440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6817","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6819","nonce":627255440}]},"public_addr":"10.20.193.196:6812/627255440","cluster_addr":"10.20.193.196:6816/627255440","heartbeat_back_addr":"10.20.193.196:6823/627255440","heartbeat_front_addr":"10.20.193.196:6819/627255440","state":["exists","up"]},{"osd":5,"uuid":"b8f758d2-6548-44b8-ab83-55138eeb765e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6844","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6846","nonce":1440695253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6848","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6850","nonce":1440695253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6856","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6857","nonce":1440695253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6852","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6854","nonce":1440695253}]},"public_addr":"10.20.193.196:6846/1440695253","cluster_addr":"10.20.193.196:6850/1440695253","heartbeat_back_addr":"10.20.193.196:6857/1440695253","heartbeat_front_addr":"10.20.193.196:6854/1440695253","state":["exists","up"]},{"osd":6,"uuid":"6f0ba2be-32a4-4a70-a952-2980d9bf9801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6811","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6813","nonce":2180386325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6815","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6818","nonce":2180386325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6824","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6825","nonce":2180386325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6820","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6822","nonce":2180386325}]},"public_addr":"10.20.193.196:6813/2180386325","cluster_addr":"10.20.193.196:6818/2180386325","heartbeat_back_addr":"10.20.193.196:6825/2180386325","heartbeat_front_addr":"10.20.193.196:6822/2180386325","state":["exists","up"]},{"osd":7,"uuid":"f3ae4755-f066-4372-ae0c-f364ea536d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6828","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6831","nonce":1122662840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6835","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6839","nonce":1122662840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6849","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6853","nonce":1122662840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6842","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6845","nonce":1122662840}]},"public_addr":"10.20.193.196:6831/1122662840","cluster_addr":"10.20.193.196:6839/1122662840","heartbeat_back_addr":"10.20.193.196:6853/1122662840","heartbeat_front_addr":"10.20.193.196:6845/1122662840","state":["exists","up"]},{"osd":8,"uuid":"05cc8f5d-d979-4eb7-93c2-5484939706e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6866","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6867","nonce":2631587950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6868","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6869","nonce":2631587950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6872","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6873","nonce":2631587950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6870","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6871","nonce":2631587950}]},"public_addr":"10.20.193.196:6867/2631587950","cluster_addr":"10.20.193.196:6869/2631587950","heartbeat_back_addr":"10.20.193.196:6873/2631587950","heartbeat_front_addr":"10.20.193.196:6871/2631587950","state":["exists","up"]},{"osd":9,"uuid":"e8713fc9-8526-41b8-91e8-573218e1b699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6858","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6859","nonce":1536643720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6860","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6861","nonce":1536643720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6864","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6865","nonce":1536643720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6862","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6863","nonce":1536643720}]},"public_addr":"10.20.193.196:6859/1536643720","cluster_addr":"10.20.193.196:6861/1536643720","heartbeat_back_addr":"10.20.193.196:6865/1536643720","heartbeat_front_addr":"10.20.193.196:6863/1536643720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T11:56:59.312 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T11:56:59.312 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T11:56:59.313 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T11:56:59.313 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T11:56:59.314 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-21T11:56:59.314 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-21T11:56:59.315 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-21T11:56:59.315 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-21T11:56:59.316 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2026-02-21T11:56:59.316 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2026-02-21T11:56:59.412 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:56:59.413 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-21T11:56:59.414 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.414 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T11:56:59.415 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:56:59.416 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T11:56:59.417 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.417 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.418 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T11:56:59.419 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T11:56:59.420 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.420 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-21T11:56:59.421 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.422 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-21T11:56:59.423 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.423 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T11:56:59.424 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.425 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-21T11:56:59.426 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.426 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T11:56:59.643 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.647 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.649 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.653 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:56:59.654 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.4 2026-02-21T11:56:59.657 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.658 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.7 2026-02-21T11:56:59.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.9 2026-02-21T11:56:59.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2026-02-21T11:56:59.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.8 2026-02-21T11:56:59.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.3 2026-02-21T11:56:59.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.2 2026-02-21T11:56:59.675 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.1 2026-02-21T11:56:59.697 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.703 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:56:59.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.5 2026-02-21T11:56:59.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.6 2026-02-21T11:57:00.658 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T11:57:00.663 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-21T11:57:00.664 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-21T11:57:00.667 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T11:57:00.671 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-21T11:57:00.672 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T11:57:00.673 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T11:57:00.688 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T11:57:00.711 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T11:57:00.723 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-21T11:57:00.824 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.4 2026-02-21T11:57:00.874 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:57:00.875 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.885 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.7 2026-02-21T11:57:00.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.3 2026-02-21T11:57:00.897 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.898 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.2 2026-02-21T11:57:00.900 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.904 INFO:teuthology.orchestra.run.trial196.stdout:64424509443 2026-02-21T11:57:00.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.8 2026-02-21T11:57:00.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.1 2026-02-21T11:57:00.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.9 2026-02-21T11:57:00.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2026-02-21T11:57:00.917 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.923 INFO:teuthology.orchestra.run.trial196.stdout:64424509442 2026-02-21T11:57:00.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.6 2026-02-21T11:57:00.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.5 2026-02-21T11:57:01.839 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T11:57:01.888 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-21T11:57:01.893 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T11:57:01.900 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T11:57:01.912 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-21T11:57:01.912 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T11:57:01.915 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-21T11:57:01.918 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T11:57:01.929 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-21T11:57:01.941 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T11:57:02.004 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.4 2026-02-21T11:57:02.019 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.051 INFO:teuthology.orchestra.run.trial196.stdout:64424509445 2026-02-21T11:57:02.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.7 2026-02-21T11:57:02.064 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.101 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.103 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.111 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.1 2026-02-21T11:57:02.116 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.116 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.6 2026-02-21T11:57:02.118 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.118 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.122 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.9 2026-02-21T11:57:02.126 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.2 2026-02-21T11:57:02.130 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.0 2026-02-21T11:57:02.131 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.132 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.8 2026-02-21T11:57:02.135 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.3 2026-02-21T11:57:02.144 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.171 INFO:teuthology.orchestra.run.trial196.stdout:64424509444 2026-02-21T11:57:02.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.5 2026-02-21T11:57:02.184 DEBUG:teuthology.parallel:result is None 2026-02-21T11:57:02.185 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T11:57:02.185 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:57:02.385 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:57:02.386 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:57:02.396 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-21T11:57:00.824974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":271408,"kb_used_data":1808,"kb_used_omap":61,"kb_used_meta":269378,"kb_avail":1048304592,"statfs":{"total":1073741824000,"available":1073463902208,"internally_reserved":0,"allocated":1851392,"data_stored":596678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63011,"internal_metadata":275843549},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849193+0000","last_change":"2026-02-21T11:56:58.849268+0000","last_active":"2026-02-21T11:56:58.849193+0000","last_peered":"2026-02-21T11:56:58.849193+0000","last_clean":"2026-02-21T11:56:58.849193+0000","last_became_active":"2026-02-21T11:56:56.596919+0000","last_became_peered":"2026-02-21T11:56:56.596919+0000","last_unstale":"2026-02-21T11:56:58.849193+0000","last_undegraded":"2026-02-21T11:56:58.849193+0000","last_fullsized":"2026-02-21T11:56:58.849193+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:18:42.927278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849171+0000","last_change":"2026-02-21T11:56:58.849239+0000","last_active":"2026-02-21T11:56:58.849171+0000","last_peered":"2026-02-21T11:56:58.849171+0000","last_clean":"2026-02-21T11:56:58.849171+0000","last_became_active":"2026-02-21T11:56:56.755278+0000","last_became_peered":"2026-02-21T11:56:56.755278+0000","last_unstale":"2026-02-21T11:56:58.849171+0000","last_undegraded":"2026-02-21T11:56:58.849171+0000","last_fullsized":"2026-02-21T11:56:58.849171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:13:31.819626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849332+0000","last_change":"2026-02-21T11:56:58.849332+0000","last_active":"2026-02-21T11:56:58.849332+0000","last_peered":"2026-02-21T11:56:58.849332+0000","last_clean":"2026-02-21T11:56:58.849332+0000","last_became_active":"2026-02-21T11:56:56.596846+0000","last_became_peered":"2026-02-21T11:56:56.596846+0000","last_unstale":"2026-02-21T11:56:58.849332+0000","last_undegraded":"2026-02-21T11:56:58.849332+0000","last_fullsized":"2026-02-21T11:56:58.849332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:19:33.407704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.599032+0000","last_change":"2026-02-21T11:56:58.599032+0000","last_active":"2026-02-21T11:56:58.599032+0000","last_peered":"2026-02-21T11:56:58.599032+0000","last_clean":"2026-02-21T11:56:58.599032+0000","last_became_active":"2026-02-21T11:56:56.596982+0000","last_became_peered":"2026-02-21T11:56:56.596982+0000","last_unstale":"2026-02-21T11:56:58.599032+0000","last_undegraded":"2026-02-21T11:56:58.599032+0000","last_fullsized":"2026-02-21T11:56:58.599032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T17:42:10.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027054200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"17'1","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849317+0000","last_change":"2026-02-21T11:56:58.849317+0000","last_active":"2026-02-21T11:56:58.849317+0000","last_peered":"2026-02-21T11:56:58.849317+0000","last_clean":"2026-02-21T11:56:58.849317+0000","last_became_active":"2026-02-21T11:56:56.788453+0000","last_became_peered":"2026-02-21T11:56:56.788453+0000","last_unstale":"2026-02-21T11:56:58.849317+0000","last_undegraded":"2026-02-21T11:56:58.849317+0000","last_fullsized":"2026-02-21T11:56:58.849317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T13:17:26.795432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021438800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.601824+0000","last_change":"2026-02-21T11:56:58.601824+0000","last_active":"2026-02-21T11:56:58.601824+0000","last_peered":"2026-02-21T11:56:58.601824+0000","last_clean":"2026-02-21T11:56:58.601824+0000","last_became_active":"2026-02-21T11:56:56.596847+0000","last_became_peered":"2026-02-21T11:56:56.596847+0000","last_unstale":"2026-02-21T11:56:58.601824+0000","last_undegraded":"2026-02-21T11:56:58.601824+0000","last_fullsized":"2026-02-21T11:56:58.601824+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:00:48.694598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849003+0000","last_change":"2026-02-21T11:56:58.849003+0000","last_active":"2026-02-21T11:56:58.849003+0000","last_peered":"2026-02-21T11:56:58.849003+0000","last_clean":"2026-02-21T11:56:58.849003+0000","last_became_active":"2026-02-21T11:56:56.596848+0000","last_became_peered":"2026-02-21T11:56:56.596848+0000","last_unstale":"2026-02-21T11:56:58.849003+0000","last_undegraded":"2026-02-21T11:56:58.849003+0000","last_fullsized":"2026-02-21T11:56:58.849003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:23:00.670053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.827792+0000","last_change":"2026-02-21T11:56:58.827792+0000","last_active":"2026-02-21T11:56:58.827792+0000","last_peered":"2026-02-21T11:56:58.827792+0000","last_clean":"2026-02-21T11:56:58.827792+0000","last_became_active":"2026-02-21T11:56:56.596965+0000","last_became_peered":"2026-02-21T11:56:56.596965+0000","last_unstale":"2026-02-21T11:56:58.827792+0000","last_undegraded":"2026-02-21T11:56:58.827792+0000","last_fullsized":"2026-02-21T11:56:58.827792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:23:00.670053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025061499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6173,"internal_metadata":27584483},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27144,"kb_used_data":184,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830456,"statfs":{"total":107374182400,"available":107346386944,"internally_reserved":0,"allocated":188416,"data_stored":59683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6820,"internal_metadata":27583836},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6166,"internal_metadata":27584490},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":4,"kb_used_meta":26939,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4873,"internal_metadata":27585783},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27144,"kb_used_data":184,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830456,"statfs":{"total":107374182400,"available":107346386944,"internally_reserved":0,"allocated":188416,"data_stored":59683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:57:02.398 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:57:02.562 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:57:02.563 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:57:02.573 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-21T11:57:00.824974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":271408,"kb_used_data":1808,"kb_used_omap":61,"kb_used_meta":269378,"kb_avail":1048304592,"statfs":{"total":1073741824000,"available":1073463902208,"internally_reserved":0,"allocated":1851392,"data_stored":596678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63011,"internal_metadata":275843549},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849193+0000","last_change":"2026-02-21T11:56:58.849268+0000","last_active":"2026-02-21T11:56:58.849193+0000","last_peered":"2026-02-21T11:56:58.849193+0000","last_clean":"2026-02-21T11:56:58.849193+0000","last_became_active":"2026-02-21T11:56:56.596919+0000","last_became_peered":"2026-02-21T11:56:56.596919+0000","last_unstale":"2026-02-21T11:56:58.849193+0000","last_undegraded":"2026-02-21T11:56:58.849193+0000","last_fullsized":"2026-02-21T11:56:58.849193+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:18:42.927278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849171+0000","last_change":"2026-02-21T11:56:58.849239+0000","last_active":"2026-02-21T11:56:58.849171+0000","last_peered":"2026-02-21T11:56:58.849171+0000","last_clean":"2026-02-21T11:56:58.849171+0000","last_became_active":"2026-02-21T11:56:56.755278+0000","last_became_peered":"2026-02-21T11:56:56.755278+0000","last_unstale":"2026-02-21T11:56:58.849171+0000","last_undegraded":"2026-02-21T11:56:58.849171+0000","last_fullsized":"2026-02-21T11:56:58.849171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:13:31.819626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849332+0000","last_change":"2026-02-21T11:56:58.849332+0000","last_active":"2026-02-21T11:56:58.849332+0000","last_peered":"2026-02-21T11:56:58.849332+0000","last_clean":"2026-02-21T11:56:58.849332+0000","last_became_active":"2026-02-21T11:56:56.596846+0000","last_became_peered":"2026-02-21T11:56:56.596846+0000","last_unstale":"2026-02-21T11:56:58.849332+0000","last_undegraded":"2026-02-21T11:56:58.849332+0000","last_fullsized":"2026-02-21T11:56:58.849332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:19:33.407704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.599032+0000","last_change":"2026-02-21T11:56:58.599032+0000","last_active":"2026-02-21T11:56:58.599032+0000","last_peered":"2026-02-21T11:56:58.599032+0000","last_clean":"2026-02-21T11:56:58.599032+0000","last_became_active":"2026-02-21T11:56:56.596982+0000","last_became_peered":"2026-02-21T11:56:56.596982+0000","last_unstale":"2026-02-21T11:56:58.599032+0000","last_undegraded":"2026-02-21T11:56:58.599032+0000","last_fullsized":"2026-02-21T11:56:58.599032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T17:42:10.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027054200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"17'1","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849317+0000","last_change":"2026-02-21T11:56:58.849317+0000","last_active":"2026-02-21T11:56:58.849317+0000","last_peered":"2026-02-21T11:56:58.849317+0000","last_clean":"2026-02-21T11:56:58.849317+0000","last_became_active":"2026-02-21T11:56:56.788453+0000","last_became_peered":"2026-02-21T11:56:56.788453+0000","last_unstale":"2026-02-21T11:56:58.849317+0000","last_undegraded":"2026-02-21T11:56:58.849317+0000","last_fullsized":"2026-02-21T11:56:58.849317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T13:17:26.795432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00021438800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.601824+0000","last_change":"2026-02-21T11:56:58.601824+0000","last_active":"2026-02-21T11:56:58.601824+0000","last_peered":"2026-02-21T11:56:58.601824+0000","last_clean":"2026-02-21T11:56:58.601824+0000","last_became_active":"2026-02-21T11:56:56.596847+0000","last_became_peered":"2026-02-21T11:56:56.596847+0000","last_unstale":"2026-02-21T11:56:58.601824+0000","last_undegraded":"2026-02-21T11:56:58.601824+0000","last_fullsized":"2026-02-21T11:56:58.601824+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:00:48.694598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.849003+0000","last_change":"2026-02-21T11:56:58.849003+0000","last_active":"2026-02-21T11:56:58.849003+0000","last_peered":"2026-02-21T11:56:58.849003+0000","last_clean":"2026-02-21T11:56:58.849003+0000","last_became_active":"2026-02-21T11:56:56.596848+0000","last_became_peered":"2026-02-21T11:56:56.596848+0000","last_unstale":"2026-02-21T11:56:58.849003+0000","last_undegraded":"2026-02-21T11:56:58.849003+0000","last_fullsized":"2026-02-21T11:56:58.849003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:23:00.670053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T11:56:58.827792+0000","last_change":"2026-02-21T11:56:58.827792+0000","last_active":"2026-02-21T11:56:58.827792+0000","last_peered":"2026-02-21T11:56:58.827792+0000","last_clean":"2026-02-21T11:56:58.827792+0000","last_became_active":"2026-02-21T11:56:56.596965+0000","last_became_peered":"2026-02-21T11:56:56.596965+0000","last_unstale":"2026-02-21T11:56:58.827792+0000","last_undegraded":"2026-02-21T11:56:58.827792+0000","last_fullsized":"2026-02-21T11:56:58.827792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:23:00.670053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025061499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6173,"internal_metadata":27584483},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27144,"kb_used_data":184,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830456,"statfs":{"total":107374182400,"available":107346386944,"internally_reserved":0,"allocated":188416,"data_stored":59683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6820,"internal_metadata":27583836},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6166,"internal_metadata":27584490},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":4,"kb_used_meta":26939,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4873,"internal_metadata":27585783},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":180,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":184320,"data_stored":59664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27144,"kb_used_data":184,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830456,"statfs":{"total":107374182400,"available":107346386944,"internally_reserved":0,"allocated":188416,"data_stored":59683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:57:02.575 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T11:57:02.575 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T11:57:02.575 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T11:57:02.575 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T11:57:02.760 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:57:02.761 INFO:teuthology.orchestra.run.trial196.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T11:57:02.771 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T11:57:02.771 INFO:teuthology.run_tasks:Running task workunit... 2026-02-21T11:57:02.790 INFO:tasks.workunit:Pulling workunits from ref f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:57:02.791 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-21T11:57:02.791 INFO:tasks.workunit:timeout=3h 2026-02-21T11:57:02.791 INFO:tasks.workunit:cleanup=True 2026-02-21T11:57:02.791 DEBUG:teuthology.orchestra.run.trial196:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-21T11:57:02.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:57:02.796 INFO:teuthology.orchestra.run.trial196.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-21T11:57:02.796 DEBUG:teuthology.orchestra.run.trial196:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T11:57:02.844 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-21T11:57:02.844 DEBUG:teuthology.orchestra.run.trial196:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-21T11:57:02.892 DEBUG:teuthology.orchestra.run.trial196:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:57:02.942 INFO:tasks.workunit.client.0.trial196.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:Note: switching to 'f7273f3049146bdd7fa7b4a400114872a50228e9'. 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:state without impacting any branches by switching back to a branch. 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: git switch -c 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:Or undo this operation with: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: git switch - 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr: 2026-02-21T11:57:25.024 INFO:tasks.workunit.client.0.trial196.stderr:HEAD is now at f7273f30491 Merge branch 'fs_mirror_daemon_status' of https://github.com/kotreshhr/ceph into wip-bharath21-testing-2026-02-20-1039 2026-02-21T11:57:25.033 DEBUG:teuthology.orchestra.run.trial196:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-21T11:57:25.080 INFO:tasks.workunit.client.0.trial196.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-21T11:57:25.082 INFO:tasks.workunit.client.0.trial196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-21T11:57:25.082 INFO:tasks.workunit.client.0.trial196.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-21T11:57:25.179 INFO:tasks.workunit.client.0.trial196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-21T11:57:25.211 INFO:tasks.workunit.client.0.trial196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-21T11:57:25.239 INFO:tasks.workunit.client.0.trial196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-21T11:57:25.240 INFO:tasks.workunit.client.0.trial196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-21T11:57:25.240 INFO:tasks.workunit.client.0.trial196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-21T11:57:25.263 INFO:tasks.workunit.client.0.trial196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-21T11:57:25.267 DEBUG:teuthology.orchestra.run.trial196:> set -ex 2026-02-21T11:57:25.267 DEBUG:teuthology.orchestra.run.trial196:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-21T11:57:25.312 INFO:tasks.workunit:Running workunits matching rados/test_health_warnings.sh on client.0... 2026-02-21T11:57:25.314 INFO:tasks.workunit:Running workunit rados/test_health_warnings.sh... 2026-02-21T11:57:25.314 DEBUG:teuthology.orchestra.run.trial196:workunit test rados/test_health_warnings.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7273f3049146bdd7fa7b4a400114872a50228e9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_health_warnings.sh 2026-02-21T11:57:25.359 INFO:tasks.workunit.client.0.trial196.stderr:+ crushtool -o crushmap --build --num_osds 10 host straw 2 rack straw 2 row straw 2 root straw 0 2026-02-21T11:57:25.369 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd setcrushmap -i crushmap 2026-02-21T11:57:26.626 INFO:tasks.workunit.client.0.trial196.stderr:6 2026-02-21T11:57:26.640 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd tree 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout:-11 10.00000 root root 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: -9 8.00000 row row0 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: -6 4.00000 rack rack0 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: -1 2.00000 host host0 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: 0 1.00000 osd.0 up 1.00000 1.00000 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: 1 1.00000 osd.1 up 1.00000 1.00000 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: -2 2.00000 host host1 2026-02-21T11:57:26.844 INFO:tasks.workunit.client.0.trial196.stdout: 2 1.00000 osd.2 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 3 1.00000 osd.3 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: -7 4.00000 rack rack1 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: -3 2.00000 host host2 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 4 1.00000 osd.4 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 5 1.00000 osd.5 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: -4 2.00000 host host3 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 6 1.00000 osd.6 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 7 1.00000 osd.7 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout:-10 2.00000 row row1 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: -8 2.00000 rack rack2 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: -5 2.00000 host host4 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 8 1.00000 osd.8 up 1.00000 1.00000 2026-02-21T11:57:26.845 INFO:tasks.workunit.client.0.trial196.stdout: 9 1.00000 osd.9 up 1.00000 1.00000 2026-02-21T11:57:26.854 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph tell 'osd.*' injectargs --osd_max_markdown_count 1024 --osd_max_markdown_period 1 2026-02-21T11:57:26.946 INFO:tasks.workunit.client.0.trial196.stdout:osd.0: {} 2026-02-21T11:57:26.963 INFO:tasks.workunit.client.0.trial196.stdout:osd.1: {} 2026-02-21T11:57:26.980 INFO:tasks.workunit.client.0.trial196.stdout:osd.2: {} 2026-02-21T11:57:26.997 INFO:tasks.workunit.client.0.trial196.stdout:osd.3: {} 2026-02-21T11:57:27.014 INFO:tasks.workunit.client.0.trial196.stdout:osd.4: {} 2026-02-21T11:57:27.031 INFO:tasks.workunit.client.0.trial196.stdout:osd.5: {} 2026-02-21T11:57:27.048 INFO:tasks.workunit.client.0.trial196.stdout:osd.6: {} 2026-02-21T11:57:27.065 INFO:tasks.workunit.client.0.trial196.stdout:osd.7: {} 2026-02-21T11:57:27.082 INFO:tasks.workunit.client.0.trial196.stdout:osd.8: {} 2026-02-21T11:57:27.099 INFO:tasks.workunit.client.0.trial196.stdout:osd.9: {} 2026-02-21T11:57:27.108 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd set noout 2026-02-21T11:57:28.633 INFO:tasks.workunit.client.0.trial196.stderr:noout is set 2026-02-21T11:57:28.644 INFO:tasks.workunit.client.0.trial196.stderr:+ test_mark_two_osds_same_host_down 2026-02-21T11:57:28.644 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd set noup 2026-02-21T11:57:30.635 INFO:tasks.workunit.client.0.trial196.stderr:noup is set 2026-02-21T11:57:30.647 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd down osd.0 osd.1 2026-02-21T11:57:30.801 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:30.792+0000 7fd14b396640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2026-02-21T11:57:31.691 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:31.680+0000 7fd14b396640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2026-02-21T11:57:31.691 INFO:tasks.workunit.client.0.trial196.stderr:osd.0 is already down. osd.1 is already down. 2026-02-21T11:57:31.701 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:31.983 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T11:57:31.983 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T11:57:31.983 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_DOWN: 2 osds down 2026-02-21T11:57:31.983 INFO:tasks.workunit.client.0.trial196.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:31.983 INFO:tasks.workunit.client.0.trial196.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:31.983 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2026-02-21T11:57:31.984 INFO:tasks.workunit.client.0.trial196.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T11:57:31.993 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:31.993 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '1 host' 2026-02-21T11:57:32.258 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T11:57:32.258 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:32.258 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '2 osds' 2026-02-21T11:57:32.502 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T11:57:32.503 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:32.503 INFO:tasks.workunit.client.0.trial196.stderr:+ grep osd.0 2026-02-21T11:57:32.781 INFO:tasks.workunit.client.0.trial196.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:32.781 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:32.782 INFO:tasks.workunit.client.0.trial196.stderr:+ grep osd.1 2026-02-21T11:57:32.912 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:57:32.900+0000 7fa9b2033640 -1 osd.0 28 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:32.970 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:57:32.960+0000 7fbe2f1ea640 -1 osd.1 28 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:33.064 INFO:tasks.workunit.client.0.trial196.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:33.064 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd unset noup 2026-02-21T11:57:33.645 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:57:33.636+0000 7fa9a8f01640 -1 osd.0 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:33.646 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:57:33.636+0000 7fbe260b8640 -1 osd.1 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:34.645 INFO:tasks.workunit.client.0.trial196.stderr:noup is unset 2026-02-21T11:57:34.657 INFO:tasks.workunit.client.0.trial196.stderr:+ wait_for_healthy 2026-02-21T11:57:34.657 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:34.657 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:57:34.910 INFO:tasks.workunit.client.0.trial196.stderr:+ test_mark_two_osds_same_rack_down 2026-02-21T11:57:34.911 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd set noup 2026-02-21T11:57:36.648 INFO:tasks.workunit.client.0.trial196.stderr:noup is set 2026-02-21T11:57:36.662 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd down osd.8 osd.9 2026-02-21T11:57:36.815 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:36.804+0000 7fd14b396640 -1 mon.a@0(leader).osd e32 definitely_dead 0 2026-02-21T11:57:37.704 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:37.696+0000 7fd14b396640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2026-02-21T11:57:37.704 INFO:tasks.workunit.client.0.trial196.stderr:osd.8 is already down. osd.9 is already down. 2026-02-21T11:57:37.715 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:37.973 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_DOWN: 2 osds down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout: host host4 (root=root,row=row1,rack=rack2) (2 osds) is down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_RACK_DOWN: 1 rack (2 osds) down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout: rack rack2 (root=root,row=row1) (2 osds) is down 2026-02-21T11:57:37.974 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_ROW_DOWN: 1 row (2 osds) down 2026-02-21T11:57:37.975 INFO:tasks.workunit.client.0.trial196.stdout: row row1 (root=root) (2 osds) is down 2026-02-21T11:57:37.985 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:37.985 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '1 host' 2026-02-21T11:57:38.228 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T11:57:38.228 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:38.228 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '1 rack' 2026-02-21T11:57:38.478 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T11:57:38.479 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:38.479 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '1 row' 2026-02-21T11:57:38.725 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T11:57:38.726 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:38.726 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '2 osds' 2026-02-21T11:57:38.772 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:57:38.764+0000 7f5d780e0640 -1 osd.8 34 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:38.974 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:57:38.968+0000 7fddc78f6640 -1 osd.9 34 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:38.975 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T11:57:38.976 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:38.976 INFO:tasks.workunit.client.0.trial196.stderr:+ grep osd.8 2026-02-21T11:57:39.252 INFO:tasks.workunit.client.0.trial196.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T11:57:39.253 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:39.253 INFO:tasks.workunit.client.0.trial196.stderr:+ grep osd.9 2026-02-21T11:57:39.519 INFO:tasks.workunit.client.0.trial196.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T11:57:39.519 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd unset noup 2026-02-21T11:57:39.659 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:57:39.652+0000 7fddb647a640 -1 osd.9 35 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:39.659 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:57:39.652+0000 7f5d6ac6c640 -1 osd.8 35 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:40.659 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:57:40.652+0000 7f5d72fb6640 -1 osd.8 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:40.660 INFO:tasks.ceph.osd.9.trial196.stderr:2026-02-21T11:57:40.652+0000 7fddbe7c4640 -1 osd.9 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:41.661 INFO:tasks.workunit.client.0.trial196.stderr:noup is unset 2026-02-21T11:57:41.674 INFO:tasks.workunit.client.0.trial196.stderr:+ wait_for_healthy 2026-02-21T11:57:41.674 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:41.674 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:57:41.923 INFO:tasks.workunit.client.0.trial196.stderr:+ test_mark_all_but_last_osds_down 2026-02-21T11:57:41.924 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd set noup 2026-02-21T11:57:43.665 INFO:tasks.workunit.client.0.trial196.stderr:noup is set 2026-02-21T11:57:43.680 INFO:tasks.workunit.client.0.trial196.stderr:++ ceph osd ls 2026-02-21T11:57:43.680 INFO:tasks.workunit.client.0.trial196.stderr:++ sed '$d' 2026-02-21T11:57:43.894 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd down 0 1 2 3 4 5 6 7 8 2026-02-21T11:57:44.053 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:44.048+0000 7fd14b396640 -1 mon.a@0(leader).osd e39 definitely_dead 0 2026-02-21T11:57:44.721 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:44.716+0000 7fd14b396640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2026-02-21T11:57:44.721 INFO:tasks.workunit.client.0.trial196.stderr:osd.0 is already down. osd.1 is already down. osd.2 is already down. osd.3 is already down. osd.4 is already down. osd.5 is already down. osd.6 is already down. osd.7 is already down. osd.8 is already down. 2026-02-21T11:57:44.731 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:45.009 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_DOWN: 9 osds down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.2 (root=root,row=row0,rack=rack0,host=host1) is down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.3 (root=root,row=row0,rack=rack0,host=host1) is down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.4 (root=root,row=row0,rack=rack1,host=host2) is down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.5 (root=root,row=row0,rack=rack1,host=host2) is down 2026-02-21T11:57:45.010 INFO:tasks.workunit.client.0.trial196.stdout: osd.6 (root=root,row=row0,rack=rack1,host=host3) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: osd.7 (root=root,row=row0,rack=rack1,host=host3) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_HOST_DOWN: 4 hosts (8 osds) down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: host host1 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: host host2 (root=root,row=row0,rack=rack1) (2 osds) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: host host3 (root=root,row=row0,rack=rack1) (2 osds) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_RACK_DOWN: 2 racks (8 osds) down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: rack rack0 (root=root,row=row0) (4 osds) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout: rack rack1 (root=root,row=row0) (4 osds) is down 2026-02-21T11:57:45.011 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_ROW_DOWN: 1 row (8 osds) down 2026-02-21T11:57:45.012 INFO:tasks.workunit.client.0.trial196.stdout: row row0 (root=root) (8 osds) is down 2026-02-21T11:57:45.019 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:45.019 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '1 row' 2026-02-21T11:57:45.285 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T11:57:45.286 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:45.286 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '2 racks' 2026-02-21T11:57:45.527 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T11:57:45.527 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:45.528 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '4 hosts' 2026-02-21T11:57:45.773 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T11:57:45.774 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:45.774 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '9 osds' 2026-02-21T11:57:46.022 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T11:57:46.023 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd unset noup 2026-02-21T11:57:46.037 INFO:tasks.ceph.osd.6.trial196.stderr:2026-02-21T11:57:46.032+0000 7f7182186640 -1 osd.6 41 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:46.096 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:57:46.088+0000 7fbe2e9e9640 -1 osd.1 41 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:46.501 INFO:tasks.ceph.osd.5.trial196.stderr:2026-02-21T11:57:46.496+0000 7f57aaf2d640 -1 osd.5 41 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:46.674 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:57:46.668+0000 7fbe260b8640 -1 osd.1 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:46.674 INFO:tasks.ceph.osd.5.trial196.stderr:2026-02-21T11:57:46.668+0000 7f57a5e03640 -1 osd.5 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:46.674 INFO:tasks.ceph.osd.6.trial196.stderr:2026-02-21T11:57:46.668+0000 7f7179054640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:46.771 INFO:tasks.ceph.osd.8.trial196.stderr:2026-02-21T11:57:46.764+0000 7f5d72fb6640 -1 osd.8 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:46.776 INFO:tasks.ceph.osd.7.trial196.stderr:2026-02-21T11:57:46.768+0000 7f2307696640 -1 osd.7 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:46.795 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:57:46.788+0000 7fa9a8f01640 -1 osd.0 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:46.954 INFO:tasks.ceph.osd.4.trial196.stderr:2026-02-21T11:57:46.948+0000 7f0f14403640 -1 osd.4 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:47.022 INFO:tasks.ceph.osd.2.trial196.stderr:2026-02-21T11:57:47.016+0000 7f61132d1640 -1 osd.2 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:47.154 INFO:tasks.ceph.osd.3.trial196.stderr:2026-02-21T11:57:47.148+0000 7faa379dc640 -1 osd.3 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:47.678 INFO:tasks.workunit.client.0.trial196.stderr:noup is unset 2026-02-21T11:57:47.694 INFO:tasks.workunit.client.0.trial196.stderr:+ wait_for_healthy 2026-02-21T11:57:47.694 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:47.694 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:57:47.931 INFO:tasks.workunit.client.0.trial196.stderr:+ test_mark_two_osds_same_host_down_with_classes 2026-02-21T11:57:47.931 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd set noup 2026-02-21T11:57:49.684 INFO:tasks.workunit.client.0.trial196.stderr:noup is set 2026-02-21T11:57:49.696 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd crush set-device-class ssd osd.0 osd.2 osd.4 osd.6 osd.8 2026-02-21T11:57:51.689 INFO:tasks.workunit.client.0.trial196.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. osd.2 already set to class ssd. set-device-class item id 2 name 'osd.2' device_class 'ssd': no change. osd.4 already set to class ssd. set-device-class item id 4 name 'osd.4' device_class 'ssd': no change. osd.6 already set to class ssd. set-device-class item id 6 name 'osd.6' device_class 'ssd': no change. osd.8 already set to class ssd. set-device-class item id 8 name 'osd.8' device_class 'ssd': no change. set osd(s) to class 'ssd' 2026-02-21T11:57:51.701 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd crush set-device-class hdd osd.1 osd.3 osd.5 osd.7 osd.9 2026-02-21T11:57:53.694 INFO:tasks.workunit.client.0.trial196.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. osd.3 already set to class hdd. set-device-class item id 3 name 'osd.3' device_class 'hdd': no change. osd.5 already set to class hdd. set-device-class item id 5 name 'osd.5' device_class 'hdd': no change. osd.7 already set to class hdd. set-device-class item id 7 name 'osd.7' device_class 'hdd': no change. osd.9 already set to class hdd. set-device-class item id 9 name 'osd.9' device_class 'hdd': no change. set osd(s) to class 'hdd' 2026-02-21T11:57:53.707 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd down osd.0 osd.1 2026-02-21T11:57:53.856 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:53.848+0000 7fd14b396640 -1 mon.a@0(leader).osd e49 definitely_dead 0 2026-02-21T11:57:54.749 INFO:tasks.ceph.mon.a.trial196.stderr:2026-02-21T11:57:54.744+0000 7fd14b396640 -1 mon.a@0(leader).osd e50 definitely_dead 0 2026-02-21T11:57:54.749 INFO:tasks.workunit.client.0.trial196.stderr:osd.0 is already down. osd.1 is already down. 2026-02-21T11:57:54.759 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_DOWN: 2 osds down 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2026-02-21T11:57:55.026 INFO:tasks.workunit.client.0.trial196.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T11:57:55.038 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:55.038 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '1 host' 2026-02-21T11:57:55.288 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T11:57:55.289 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:55.289 INFO:tasks.workunit.client.0.trial196.stderr:+ grep '2 osds' 2026-02-21T11:57:55.541 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T11:57:55.542 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:55.542 INFO:tasks.workunit.client.0.trial196.stderr:+ grep osd.0 2026-02-21T11:57:55.812 INFO:tasks.workunit.client.0.trial196.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:55.812 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health detail 2026-02-21T11:57:55.812 INFO:tasks.workunit.client.0.trial196.stderr:+ grep osd.1 2026-02-21T11:57:55.861 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:57:55.856+0000 7fa9b2033640 -1 osd.0 51 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T11:57:56.077 INFO:tasks.workunit.client.0.trial196.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T11:57:56.077 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph osd unset noup 2026-02-21T11:57:56.703 INFO:tasks.ceph.osd.0.trial196.stderr:2026-02-21T11:57:56.696+0000 7fa9a8f01640 -1 osd.0 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:57:57.705 INFO:tasks.workunit.client.0.trial196.stderr:noup is unset 2026-02-21T11:57:57.719 INFO:tasks.workunit.client.0.trial196.stderr:+ wait_for_healthy 2026-02-21T11:57:57.719 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:57.719 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:57:57.959 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noout flag(s) set; 1 osds down 2026-02-21T11:57:57.960 INFO:tasks.workunit.client.0.trial196.stderr:+ sleep 1 2026-02-21T11:57:58.961 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:57:58.962 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:57:59.211 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noout flag(s) set; 1 osds down 2026-02-21T11:57:59.211 INFO:tasks.workunit.client.0.trial196.stderr:+ sleep 1 2026-02-21T11:58:00.213 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:58:00.214 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:58:00.256 INFO:tasks.ceph.osd.1.trial196.stderr:2026-02-21T11:58:00.248+0000 7fbe260b8640 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T11:58:00.453 INFO:tasks.workunit.client.0.trial196.stdout:HEALTH_WARN noout flag(s) set; 1 osds down 2026-02-21T11:58:00.454 INFO:tasks.workunit.client.0.trial196.stderr:+ sleep 1 2026-02-21T11:58:01.455 INFO:tasks.workunit.client.0.trial196.stderr:+ ceph health 2026-02-21T11:58:01.455 INFO:tasks.workunit.client.0.trial196.stderr:+ grep down 2026-02-21T11:58:01.702 INFO:tasks.workunit.client.0.trial196.stderr:+ exit 0 2026-02-21T11:58:01.703 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T11:58:01.703 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T11:58:01.753 INFO:tasks.workunit:Stopping ['rados/test_health_warnings.sh'] on client.0... 2026-02-21T11:58:01.753 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-21T11:58:02.121 DEBUG:teuthology.parallel:result is None 2026-02-21T11:58:02.121 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T11:58:02.128 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T11:58:02.128 DEBUG:teuthology.orchestra.run.trial196:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T11:58:02.176 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T11:58:02.176 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T11:58:02.228 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T11:58:02.229 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:02.384 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:02.384 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:02.395 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2026-02-21T11:58:01.713756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275728,"kb_used_data":6128,"kb_used_omap":265,"kb_used_meta":269174,"kb_avail":1048300272,"statfs":{"total":1073741824000,"available":1073459478528,"internally_reserved":0,"allocated":6275072,"data_stored":3038299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":272208,"internal_metadata":275634352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012873"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709420+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:57:59.709420+0000","last_peered":"2026-02-21T11:57:59.709420+0000","last_clean":"2026-02-21T11:57:59.709420+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:57:59.709420+0000","last_undegraded":"2026-02-21T11:57:59.709420+0000","last_fullsized":"2026-02-21T11:57:59.709420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709353+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:57:59.709353+0000","last_peered":"2026-02-21T11:57:59.709353+0000","last_clean":"2026-02-21T11:57:59.709353+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:57:59.709353+0000","last_undegraded":"2026-02-21T11:57:59.709353+0000","last_fullsized":"2026-02-21T11:57:59.709353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709448+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:57:59.709448+0000","last_peered":"2026-02-21T11:57:59.709448+0000","last_clean":"2026-02-21T11:57:59.709448+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:57:59.709448+0000","last_undegraded":"2026-02-21T11:57:59.709448+0000","last_fullsized":"2026-02-21T11:57:59.709448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":55,"state":"active+undersized","last_fresh":"2026-02-21T11:57:59.709410+0000","last_change":"2026-02-21T11:57:59.709410+0000","last_active":"2026-02-21T11:57:59.709410+0000","last_peered":"2026-02-21T11:57:59.709410+0000","last_clean":"2026-02-21T11:57:44.298585+0000","last_became_active":"2026-02-21T11:57:55.697563+0000","last_became_peered":"2026-02-21T11:57:55.697563+0000","last_unstale":"2026-02-21T11:57:59.709410+0000","last_undegraded":"2026-02-21T11:57:59.709410+0000","last_fullsized":"2026-02-21T11:57:55.697332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":90,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709482+0000","last_change":"2026-02-21T11:57:59.709482+0000","last_active":"2026-02-21T11:57:59.709482+0000","last_peered":"2026-02-21T11:57:59.709482+0000","last_clean":"2026-02-21T11:57:55.285763+0000","last_became_active":"2026-02-21T11:57:55.696907+0000","last_became_peered":"2026-02-21T11:57:55.696907+0000","last_unstale":"2026-02-21T11:57:59.709482+0000","last_undegraded":"2026-02-21T11:57:55.696616+0000","last_fullsized":"2026-02-21T11:57:55.696557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":93,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709405+0000","last_change":"2026-02-21T11:57:59.709405+0000","last_active":"2026-02-21T11:57:59.709405+0000","last_peered":"2026-02-21T11:57:59.709405+0000","last_clean":"2026-02-21T11:57:55.285632+0000","last_became_active":"2026-02-21T11:57:55.697748+0000","last_became_peered":"2026-02-21T11:57:55.697748+0000","last_unstale":"2026-02-21T11:57:59.709405+0000","last_undegraded":"2026-02-21T11:57:55.697396+0000","last_fullsized":"2026-02-21T11:57:55.697329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.983506+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:57:59.983506+0000","last_peered":"2026-02-21T11:57:59.983506+0000","last_clean":"2026-02-21T11:57:59.983506+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:57:59.983506+0000","last_undegraded":"2026-02-21T11:57:59.983506+0000","last_fullsized":"2026-02-21T11:57:59.983506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709428+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:57:59.709428+0000","last_peered":"2026-02-21T11:57:59.709428+0000","last_clean":"2026-02-21T11:57:59.709428+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:57:59.709428+0000","last_undegraded":"2026-02-21T11:57:59.709428+0000","last_fullsized":"2026-02-21T11:57:59.709428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27747,"internal_metadata":27562909},"hb_peers":[0,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":296103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26435,"internal_metadata":27564221},"hb_peers":[0,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27740,"internal_metadata":27562916},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25787,"internal_metadata":27564869},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":16,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":26,"kb_used_meta":26917,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27091,"internal_metadata":27563565},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27584,"kb_used_data":624,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830016,"statfs":{"total":107374182400,"available":107345936384,"internally_reserved":0,"allocated":638976,"data_stored":311862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:02.396 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:02.554 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:02.554 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:02.566 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2026-02-21T11:58:01.713756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275728,"kb_used_data":6128,"kb_used_omap":265,"kb_used_meta":269174,"kb_avail":1048300272,"statfs":{"total":1073741824000,"available":1073459478528,"internally_reserved":0,"allocated":6275072,"data_stored":3038299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":272208,"internal_metadata":275634352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012873"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709420+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:57:59.709420+0000","last_peered":"2026-02-21T11:57:59.709420+0000","last_clean":"2026-02-21T11:57:59.709420+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:57:59.709420+0000","last_undegraded":"2026-02-21T11:57:59.709420+0000","last_fullsized":"2026-02-21T11:57:59.709420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709353+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:57:59.709353+0000","last_peered":"2026-02-21T11:57:59.709353+0000","last_clean":"2026-02-21T11:57:59.709353+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:57:59.709353+0000","last_undegraded":"2026-02-21T11:57:59.709353+0000","last_fullsized":"2026-02-21T11:57:59.709353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709448+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:57:59.709448+0000","last_peered":"2026-02-21T11:57:59.709448+0000","last_clean":"2026-02-21T11:57:59.709448+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:57:59.709448+0000","last_undegraded":"2026-02-21T11:57:59.709448+0000","last_fullsized":"2026-02-21T11:57:59.709448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":55,"state":"active+undersized","last_fresh":"2026-02-21T11:57:59.709410+0000","last_change":"2026-02-21T11:57:59.709410+0000","last_active":"2026-02-21T11:57:59.709410+0000","last_peered":"2026-02-21T11:57:59.709410+0000","last_clean":"2026-02-21T11:57:44.298585+0000","last_became_active":"2026-02-21T11:57:55.697563+0000","last_became_peered":"2026-02-21T11:57:55.697563+0000","last_unstale":"2026-02-21T11:57:59.709410+0000","last_undegraded":"2026-02-21T11:57:59.709410+0000","last_fullsized":"2026-02-21T11:57:55.697332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":90,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709482+0000","last_change":"2026-02-21T11:57:59.709482+0000","last_active":"2026-02-21T11:57:59.709482+0000","last_peered":"2026-02-21T11:57:59.709482+0000","last_clean":"2026-02-21T11:57:55.285763+0000","last_became_active":"2026-02-21T11:57:55.696907+0000","last_became_peered":"2026-02-21T11:57:55.696907+0000","last_unstale":"2026-02-21T11:57:59.709482+0000","last_undegraded":"2026-02-21T11:57:55.696616+0000","last_fullsized":"2026-02-21T11:57:55.696557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":93,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709405+0000","last_change":"2026-02-21T11:57:59.709405+0000","last_active":"2026-02-21T11:57:59.709405+0000","last_peered":"2026-02-21T11:57:59.709405+0000","last_clean":"2026-02-21T11:57:55.285632+0000","last_became_active":"2026-02-21T11:57:55.697748+0000","last_became_peered":"2026-02-21T11:57:55.697748+0000","last_unstale":"2026-02-21T11:57:59.709405+0000","last_undegraded":"2026-02-21T11:57:55.697396+0000","last_fullsized":"2026-02-21T11:57:55.697329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.983506+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:57:59.983506+0000","last_peered":"2026-02-21T11:57:59.983506+0000","last_clean":"2026-02-21T11:57:59.983506+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:57:59.983506+0000","last_undegraded":"2026-02-21T11:57:59.983506+0000","last_fullsized":"2026-02-21T11:57:59.983506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709428+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:57:59.709428+0000","last_peered":"2026-02-21T11:57:59.709428+0000","last_clean":"2026-02-21T11:57:59.709428+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:57:59.709428+0000","last_undegraded":"2026-02-21T11:57:59.709428+0000","last_fullsized":"2026-02-21T11:57:59.709428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27747,"internal_metadata":27562909},"hb_peers":[0,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":296103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26435,"internal_metadata":27564221},"hb_peers":[0,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27740,"internal_metadata":27562916},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25787,"internal_metadata":27564869},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":16,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":26,"kb_used_meta":26917,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27091,"internal_metadata":27563565},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27584,"kb_used_data":624,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830016,"statfs":{"total":107374182400,"available":107345936384,"internally_reserved":0,"allocated":638976,"data_stored":311862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:02.569 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:02.733 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:02.734 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:02.746 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2026-02-21T11:58:01.713756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275728,"kb_used_data":6128,"kb_used_omap":265,"kb_used_meta":269174,"kb_avail":1048300272,"statfs":{"total":1073741824000,"available":1073459478528,"internally_reserved":0,"allocated":6275072,"data_stored":3038299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":272208,"internal_metadata":275634352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012873"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709420+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:57:59.709420+0000","last_peered":"2026-02-21T11:57:59.709420+0000","last_clean":"2026-02-21T11:57:59.709420+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:57:59.709420+0000","last_undegraded":"2026-02-21T11:57:59.709420+0000","last_fullsized":"2026-02-21T11:57:59.709420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709353+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:57:59.709353+0000","last_peered":"2026-02-21T11:57:59.709353+0000","last_clean":"2026-02-21T11:57:59.709353+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:57:59.709353+0000","last_undegraded":"2026-02-21T11:57:59.709353+0000","last_fullsized":"2026-02-21T11:57:59.709353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709448+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:57:59.709448+0000","last_peered":"2026-02-21T11:57:59.709448+0000","last_clean":"2026-02-21T11:57:59.709448+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:57:59.709448+0000","last_undegraded":"2026-02-21T11:57:59.709448+0000","last_fullsized":"2026-02-21T11:57:59.709448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":55,"state":"active+undersized","last_fresh":"2026-02-21T11:57:59.709410+0000","last_change":"2026-02-21T11:57:59.709410+0000","last_active":"2026-02-21T11:57:59.709410+0000","last_peered":"2026-02-21T11:57:59.709410+0000","last_clean":"2026-02-21T11:57:44.298585+0000","last_became_active":"2026-02-21T11:57:55.697563+0000","last_became_peered":"2026-02-21T11:57:55.697563+0000","last_unstale":"2026-02-21T11:57:59.709410+0000","last_undegraded":"2026-02-21T11:57:59.709410+0000","last_fullsized":"2026-02-21T11:57:55.697332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":90,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709482+0000","last_change":"2026-02-21T11:57:59.709482+0000","last_active":"2026-02-21T11:57:59.709482+0000","last_peered":"2026-02-21T11:57:59.709482+0000","last_clean":"2026-02-21T11:57:55.285763+0000","last_became_active":"2026-02-21T11:57:55.696907+0000","last_became_peered":"2026-02-21T11:57:55.696907+0000","last_unstale":"2026-02-21T11:57:59.709482+0000","last_undegraded":"2026-02-21T11:57:55.696616+0000","last_fullsized":"2026-02-21T11:57:55.696557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":93,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709405+0000","last_change":"2026-02-21T11:57:59.709405+0000","last_active":"2026-02-21T11:57:59.709405+0000","last_peered":"2026-02-21T11:57:59.709405+0000","last_clean":"2026-02-21T11:57:55.285632+0000","last_became_active":"2026-02-21T11:57:55.697748+0000","last_became_peered":"2026-02-21T11:57:55.697748+0000","last_unstale":"2026-02-21T11:57:59.709405+0000","last_undegraded":"2026-02-21T11:57:55.697396+0000","last_fullsized":"2026-02-21T11:57:55.697329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.983506+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:57:59.983506+0000","last_peered":"2026-02-21T11:57:59.983506+0000","last_clean":"2026-02-21T11:57:59.983506+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:57:59.983506+0000","last_undegraded":"2026-02-21T11:57:59.983506+0000","last_fullsized":"2026-02-21T11:57:59.983506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709428+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:57:59.709428+0000","last_peered":"2026-02-21T11:57:59.709428+0000","last_clean":"2026-02-21T11:57:59.709428+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:57:59.709428+0000","last_undegraded":"2026-02-21T11:57:59.709428+0000","last_fullsized":"2026-02-21T11:57:59.709428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27747,"internal_metadata":27562909},"hb_peers":[0,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":296103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26435,"internal_metadata":27564221},"hb_peers":[0,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27740,"internal_metadata":27562916},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25787,"internal_metadata":27564869},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":16,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":26,"kb_used_meta":26917,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27091,"internal_metadata":27563565},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27584,"kb_used_data":624,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830016,"statfs":{"total":107374182400,"available":107345936384,"internally_reserved":0,"allocated":638976,"data_stored":311862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:02.747 INFO:tasks.ceph.ceph_manager.ceph:PG 1.4 is not active+clean 2026-02-21T11:58:02.748 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 65, 'reported_epoch': 55, 'state': 'active+undersized', 'last_fresh': '2026-02-21T11:57:59.709410+0000', 'last_change': '2026-02-21T11:57:59.709410+0000', 'last_active': '2026-02-21T11:57:59.709410+0000', 'last_peered': '2026-02-21T11:57:59.709410+0000', 'last_clean': '2026-02-21T11:57:44.298585+0000', 'last_became_active': '2026-02-21T11:57:55.697563+0000', 'last_became_peered': '2026-02-21T11:57:55.697563+0000', 'last_unstale': '2026-02-21T11:57:59.709410+0000', 'last_undegraded': '2026-02-21T11:57:59.709410+0000', 'last_fullsized': '2026-02-21T11:57:55.697332+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 44, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_clean_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6], 'acting': [6], 'avail_no_missing': ['6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2026-02-21T11:58:02.748 INFO:tasks.ceph.ceph_manager.ceph:PG 1.3 is not active+clean 2026-02-21T11:58:02.748 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.3', 'version': "17'1", 'reported_seq': 90, 'reported_epoch': 55, 'state': 'active+undersized+degraded', 'last_fresh': '2026-02-21T11:57:59.709482+0000', 'last_change': '2026-02-21T11:57:59.709482+0000', 'last_active': '2026-02-21T11:57:59.709482+0000', 'last_peered': '2026-02-21T11:57:59.709482+0000', 'last_clean': '2026-02-21T11:57:55.285763+0000', 'last_became_active': '2026-02-21T11:57:55.696907+0000', 'last_became_peered': '2026-02-21T11:57:55.696907+0000', 'last_unstale': '2026-02-21T11:57:59.709482+0000', 'last_undegraded': '2026-02-21T11:57:55.696616+0000', 'last_fullsized': '2026-02-21T11:57:55.696557+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 44, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_clean_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3], 'acting': [3], 'avail_no_missing': ['3'], 'object_location_counts': [{'shards': '3', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T11:58:02.748 INFO:tasks.ceph.ceph_manager.ceph:PG 1.2 is not active+clean 2026-02-21T11:58:02.748 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.2', 'version': "19'2", 'reported_seq': 93, 'reported_epoch': 55, 'state': 'active+undersized+degraded', 'last_fresh': '2026-02-21T11:57:59.709405+0000', 'last_change': '2026-02-21T11:57:59.709405+0000', 'last_active': '2026-02-21T11:57:59.709405+0000', 'last_peered': '2026-02-21T11:57:59.709405+0000', 'last_clean': '2026-02-21T11:57:55.285632+0000', 'last_became_active': '2026-02-21T11:57:55.697748+0000', 'last_became_peered': '2026-02-21T11:57:55.697748+0000', 'last_unstale': '2026-02-21T11:57:59.709405+0000', 'last_undegraded': '2026-02-21T11:57:55.697396+0000', 'last_fullsized': '2026-02-21T11:57:55.697329+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 44, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_clean_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000745954, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4], 'acting': [4], 'avail_no_missing': ['4'], 'object_location_counts': [{'shards': '4', 'objects': 1}], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2026-02-21T11:58:02.748 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T11:58:02.954 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:02.954 INFO:teuthology.orchestra.run.trial196.stdout:{"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noout flag(s) set","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2/4 objects degraded (50.000%), 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":1},"osdmap":{"epoch":57,"num_osds":10,"num_up_osds":10,"osd_up_since":1771675080,"num_in_osds":10,"osd_in_since":1771675006,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+undersized+degraded","count":2},{"state_name":"active+undersized","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":282284032,"bytes_avail":1073459539968,"bytes_total":1073741824000,"degraded_objects":2,"degraded_total":4,"degraded_ratio":0.5},"fsmap":{"epoch":1,"btime":"2026-02-21T11:56:44:569344+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":4,"modified":"2026-02-21T11:57:52.830949+0000","services":{}},"progress_events":{}} 2026-02-21T11:58:02.966 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2026-02-21T11:58:02.966 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:03.131 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:03.132 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:03.142 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2026-02-21T11:58:02.832288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275728,"kb_used_data":6128,"kb_used_omap":265,"kb_used_meta":269174,"kb_avail":1048300272,"statfs":{"total":1073741824000,"available":1073459478528,"internally_reserved":0,"allocated":6275072,"data_stored":3038299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":272208,"internal_metadata":275634352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.128739"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709420+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:57:59.709420+0000","last_peered":"2026-02-21T11:57:59.709420+0000","last_clean":"2026-02-21T11:57:59.709420+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:57:59.709420+0000","last_undegraded":"2026-02-21T11:57:59.709420+0000","last_fullsized":"2026-02-21T11:57:59.709420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709353+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:57:59.709353+0000","last_peered":"2026-02-21T11:57:59.709353+0000","last_clean":"2026-02-21T11:57:59.709353+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:57:59.709353+0000","last_undegraded":"2026-02-21T11:57:59.709353+0000","last_fullsized":"2026-02-21T11:57:59.709353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709448+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:57:59.709448+0000","last_peered":"2026-02-21T11:57:59.709448+0000","last_clean":"2026-02-21T11:57:59.709448+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:57:59.709448+0000","last_undegraded":"2026-02-21T11:57:59.709448+0000","last_fullsized":"2026-02-21T11:57:59.709448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":55,"state":"active+undersized","last_fresh":"2026-02-21T11:57:59.709410+0000","last_change":"2026-02-21T11:57:59.709410+0000","last_active":"2026-02-21T11:57:59.709410+0000","last_peered":"2026-02-21T11:57:59.709410+0000","last_clean":"2026-02-21T11:57:44.298585+0000","last_became_active":"2026-02-21T11:57:55.697563+0000","last_became_peered":"2026-02-21T11:57:55.697563+0000","last_unstale":"2026-02-21T11:57:59.709410+0000","last_undegraded":"2026-02-21T11:57:59.709410+0000","last_fullsized":"2026-02-21T11:57:55.697332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":90,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709482+0000","last_change":"2026-02-21T11:57:59.709482+0000","last_active":"2026-02-21T11:57:59.709482+0000","last_peered":"2026-02-21T11:57:59.709482+0000","last_clean":"2026-02-21T11:57:55.285763+0000","last_became_active":"2026-02-21T11:57:55.696907+0000","last_became_peered":"2026-02-21T11:57:55.696907+0000","last_unstale":"2026-02-21T11:57:59.709482+0000","last_undegraded":"2026-02-21T11:57:55.696616+0000","last_fullsized":"2026-02-21T11:57:55.696557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":93,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709405+0000","last_change":"2026-02-21T11:57:59.709405+0000","last_active":"2026-02-21T11:57:59.709405+0000","last_peered":"2026-02-21T11:57:59.709405+0000","last_clean":"2026-02-21T11:57:55.285632+0000","last_became_active":"2026-02-21T11:57:55.697748+0000","last_became_peered":"2026-02-21T11:57:55.697748+0000","last_unstale":"2026-02-21T11:57:59.709405+0000","last_undegraded":"2026-02-21T11:57:55.697396+0000","last_fullsized":"2026-02-21T11:57:55.697329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.983506+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:57:59.983506+0000","last_peered":"2026-02-21T11:57:59.983506+0000","last_clean":"2026-02-21T11:57:59.983506+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:57:59.983506+0000","last_undegraded":"2026-02-21T11:57:59.983506+0000","last_fullsized":"2026-02-21T11:57:59.983506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709428+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:57:59.709428+0000","last_peered":"2026-02-21T11:57:59.709428+0000","last_clean":"2026-02-21T11:57:59.709428+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:57:59.709428+0000","last_undegraded":"2026-02-21T11:57:59.709428+0000","last_fullsized":"2026-02-21T11:57:59.709428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27747,"internal_metadata":27562909},"hb_peers":[0,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":296103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26435,"internal_metadata":27564221},"hb_peers":[0,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27740,"internal_metadata":27562916},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25787,"internal_metadata":27564869},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":16,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":26,"kb_used_meta":26917,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27091,"internal_metadata":27563565},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27584,"kb_used_data":624,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830016,"statfs":{"total":107374182400,"available":107345936384,"internally_reserved":0,"allocated":638976,"data_stored":311862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:06.144 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:06.305 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:06.305 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:06.317 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2026-02-21T11:58:04.832544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275728,"kb_used_data":6128,"kb_used_omap":265,"kb_used_meta":269174,"kb_avail":1048300272,"statfs":{"total":1073741824000,"available":1073459478528,"internally_reserved":0,"allocated":6275072,"data_stored":3038299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":272208,"internal_metadata":275634352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.122992"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709420+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:57:59.709420+0000","last_peered":"2026-02-21T11:57:59.709420+0000","last_clean":"2026-02-21T11:57:59.709420+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:57:59.709420+0000","last_undegraded":"2026-02-21T11:57:59.709420+0000","last_fullsized":"2026-02-21T11:57:59.709420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709353+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:57:59.709353+0000","last_peered":"2026-02-21T11:57:59.709353+0000","last_clean":"2026-02-21T11:57:59.709353+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:57:59.709353+0000","last_undegraded":"2026-02-21T11:57:59.709353+0000","last_fullsized":"2026-02-21T11:57:59.709353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709448+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:57:59.709448+0000","last_peered":"2026-02-21T11:57:59.709448+0000","last_clean":"2026-02-21T11:57:59.709448+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:57:59.709448+0000","last_undegraded":"2026-02-21T11:57:59.709448+0000","last_fullsized":"2026-02-21T11:57:59.709448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":55,"state":"active+undersized","last_fresh":"2026-02-21T11:57:59.709410+0000","last_change":"2026-02-21T11:57:59.709410+0000","last_active":"2026-02-21T11:57:59.709410+0000","last_peered":"2026-02-21T11:57:59.709410+0000","last_clean":"2026-02-21T11:57:44.298585+0000","last_became_active":"2026-02-21T11:57:55.697563+0000","last_became_peered":"2026-02-21T11:57:55.697563+0000","last_unstale":"2026-02-21T11:57:59.709410+0000","last_undegraded":"2026-02-21T11:57:59.709410+0000","last_fullsized":"2026-02-21T11:57:55.697332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":90,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709482+0000","last_change":"2026-02-21T11:57:59.709482+0000","last_active":"2026-02-21T11:57:59.709482+0000","last_peered":"2026-02-21T11:57:59.709482+0000","last_clean":"2026-02-21T11:57:55.285763+0000","last_became_active":"2026-02-21T11:57:55.696907+0000","last_became_peered":"2026-02-21T11:57:55.696907+0000","last_unstale":"2026-02-21T11:57:59.709482+0000","last_undegraded":"2026-02-21T11:57:55.696616+0000","last_fullsized":"2026-02-21T11:57:55.696557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":93,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709405+0000","last_change":"2026-02-21T11:57:59.709405+0000","last_active":"2026-02-21T11:57:59.709405+0000","last_peered":"2026-02-21T11:57:59.709405+0000","last_clean":"2026-02-21T11:57:55.285632+0000","last_became_active":"2026-02-21T11:57:55.697748+0000","last_became_peered":"2026-02-21T11:57:55.697748+0000","last_unstale":"2026-02-21T11:57:59.709405+0000","last_undegraded":"2026-02-21T11:57:55.697396+0000","last_fullsized":"2026-02-21T11:57:55.697329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.983506+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:57:59.983506+0000","last_peered":"2026-02-21T11:57:59.983506+0000","last_clean":"2026-02-21T11:57:59.983506+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:57:59.983506+0000","last_undegraded":"2026-02-21T11:57:59.983506+0000","last_fullsized":"2026-02-21T11:57:59.983506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709428+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:57:59.709428+0000","last_peered":"2026-02-21T11:57:59.709428+0000","last_clean":"2026-02-21T11:57:59.709428+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:57:59.709428+0000","last_undegraded":"2026-02-21T11:57:59.709428+0000","last_fullsized":"2026-02-21T11:57:59.709428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27747,"internal_metadata":27562909},"hb_peers":[0,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":296103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26435,"internal_metadata":27564221},"hb_peers":[0,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27740,"internal_metadata":27562916},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25787,"internal_metadata":27564869},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":16,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":26,"kb_used_meta":26917,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27091,"internal_metadata":27563565},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27584,"kb_used_data":624,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830016,"statfs":{"total":107374182400,"available":107345936384,"internally_reserved":0,"allocated":638976,"data_stored":311862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:06.318 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:06.473 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:06.474 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:06.484 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2026-02-21T11:58:04.832544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275728,"kb_used_data":6128,"kb_used_omap":265,"kb_used_meta":269174,"kb_avail":1048300272,"statfs":{"total":1073741824000,"available":1073459478528,"internally_reserved":0,"allocated":6275072,"data_stored":3038299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":272208,"internal_metadata":275634352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.122992"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709420+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:57:59.709420+0000","last_peered":"2026-02-21T11:57:59.709420+0000","last_clean":"2026-02-21T11:57:59.709420+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:57:59.709420+0000","last_undegraded":"2026-02-21T11:57:59.709420+0000","last_fullsized":"2026-02-21T11:57:59.709420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709353+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:57:59.709353+0000","last_peered":"2026-02-21T11:57:59.709353+0000","last_clean":"2026-02-21T11:57:59.709353+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:57:59.709353+0000","last_undegraded":"2026-02-21T11:57:59.709353+0000","last_fullsized":"2026-02-21T11:57:59.709353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709448+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:57:59.709448+0000","last_peered":"2026-02-21T11:57:59.709448+0000","last_clean":"2026-02-21T11:57:59.709448+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:57:59.709448+0000","last_undegraded":"2026-02-21T11:57:59.709448+0000","last_fullsized":"2026-02-21T11:57:59.709448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":55,"state":"active+undersized","last_fresh":"2026-02-21T11:57:59.709410+0000","last_change":"2026-02-21T11:57:59.709410+0000","last_active":"2026-02-21T11:57:59.709410+0000","last_peered":"2026-02-21T11:57:59.709410+0000","last_clean":"2026-02-21T11:57:44.298585+0000","last_became_active":"2026-02-21T11:57:55.697563+0000","last_became_peered":"2026-02-21T11:57:55.697563+0000","last_unstale":"2026-02-21T11:57:59.709410+0000","last_undegraded":"2026-02-21T11:57:59.709410+0000","last_fullsized":"2026-02-21T11:57:55.697332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":90,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709482+0000","last_change":"2026-02-21T11:57:59.709482+0000","last_active":"2026-02-21T11:57:59.709482+0000","last_peered":"2026-02-21T11:57:59.709482+0000","last_clean":"2026-02-21T11:57:55.285763+0000","last_became_active":"2026-02-21T11:57:55.696907+0000","last_became_peered":"2026-02-21T11:57:55.696907+0000","last_unstale":"2026-02-21T11:57:59.709482+0000","last_undegraded":"2026-02-21T11:57:55.696616+0000","last_fullsized":"2026-02-21T11:57:55.696557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":93,"reported_epoch":55,"state":"active+undersized+degraded","last_fresh":"2026-02-21T11:57:59.709405+0000","last_change":"2026-02-21T11:57:59.709405+0000","last_active":"2026-02-21T11:57:59.709405+0000","last_peered":"2026-02-21T11:57:59.709405+0000","last_clean":"2026-02-21T11:57:55.285632+0000","last_became_active":"2026-02-21T11:57:55.697748+0000","last_became_peered":"2026-02-21T11:57:55.697748+0000","last_unstale":"2026-02-21T11:57:59.709405+0000","last_undegraded":"2026-02-21T11:57:55.697396+0000","last_fullsized":"2026-02-21T11:57:55.697329+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.983506+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:57:59.983506+0000","last_peered":"2026-02-21T11:57:59.983506+0000","last_clean":"2026-02-21T11:57:59.983506+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:57:59.983506+0000","last_undegraded":"2026-02-21T11:57:59.983506+0000","last_fullsized":"2026-02-21T11:57:59.983506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2026-02-21T11:57:59.709428+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:57:59.709428+0000","last_peered":"2026-02-21T11:57:59.709428+0000","last_clean":"2026-02-21T11:57:59.709428+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:57:59.709428+0000","last_undegraded":"2026-02-21T11:57:59.709428+0000","last_fullsized":"2026-02-21T11:57:59.709428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27747,"internal_metadata":27562909},"hb_peers":[0,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":296103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26435,"internal_metadata":27564221},"hb_peers":[0,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27740,"internal_metadata":27562916},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25787,"internal_metadata":27564869},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27572,"kb_used_data":612,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830028,"statfs":{"total":107374182400,"available":107345948672,"internally_reserved":0,"allocated":626688,"data_stored":303787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":16,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27576,"kb_used_data":616,"kb_used_omap":26,"kb_used_meta":26917,"kb_avail":104830024,"statfs":{"total":107374182400,"available":107345944576,"internally_reserved":0,"allocated":630784,"data_stored":303806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27091,"internal_metadata":27563565},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27584,"kb_used_data":624,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830016,"statfs":{"total":107374182400,"available":107345936384,"internally_reserved":0,"allocated":638976,"data_stored":311862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:06.485 INFO:tasks.ceph.ceph_manager.ceph:PG 1.4 is not active+clean 2026-02-21T11:58:06.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 65, 'reported_epoch': 55, 'state': 'active+undersized', 'last_fresh': '2026-02-21T11:57:59.709410+0000', 'last_change': '2026-02-21T11:57:59.709410+0000', 'last_active': '2026-02-21T11:57:59.709410+0000', 'last_peered': '2026-02-21T11:57:59.709410+0000', 'last_clean': '2026-02-21T11:57:44.298585+0000', 'last_became_active': '2026-02-21T11:57:55.697563+0000', 'last_became_peered': '2026-02-21T11:57:55.697563+0000', 'last_unstale': '2026-02-21T11:57:59.709410+0000', 'last_undegraded': '2026-02-21T11:57:59.709410+0000', 'last_fullsized': '2026-02-21T11:57:55.697332+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 44, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_clean_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-22T20:26:07.458690+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6], 'acting': [6], 'avail_no_missing': ['6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2026-02-21T11:58:06.486 INFO:tasks.ceph.ceph_manager.ceph:PG 1.3 is not active+clean 2026-02-21T11:58:06.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.3', 'version': "17'1", 'reported_seq': 90, 'reported_epoch': 55, 'state': 'active+undersized+degraded', 'last_fresh': '2026-02-21T11:57:59.709482+0000', 'last_change': '2026-02-21T11:57:59.709482+0000', 'last_active': '2026-02-21T11:57:59.709482+0000', 'last_peered': '2026-02-21T11:57:59.709482+0000', 'last_clean': '2026-02-21T11:57:55.285763+0000', 'last_became_active': '2026-02-21T11:57:55.696907+0000', 'last_became_peered': '2026-02-21T11:57:55.696907+0000', 'last_unstale': '2026-02-21T11:57:59.709482+0000', 'last_undegraded': '2026-02-21T11:57:55.696616+0000', 'last_fullsized': '2026-02-21T11:57:55.696557+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 44, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_clean_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-22T12:08:03.738651+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3], 'acting': [3], 'avail_no_missing': ['3'], 'object_location_counts': [{'shards': '3', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-21T11:58:06.486 INFO:tasks.ceph.ceph_manager.ceph:PG 1.2 is not active+clean 2026-02-21T11:58:06.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.2', 'version': "19'2", 'reported_seq': 93, 'reported_epoch': 55, 'state': 'active+undersized+degraded', 'last_fresh': '2026-02-21T11:57:59.709405+0000', 'last_change': '2026-02-21T11:57:59.709405+0000', 'last_active': '2026-02-21T11:57:59.709405+0000', 'last_peered': '2026-02-21T11:57:59.709405+0000', 'last_clean': '2026-02-21T11:57:55.285632+0000', 'last_became_active': '2026-02-21T11:57:55.697748+0000', 'last_became_peered': '2026-02-21T11:57:55.697748+0000', 'last_unstale': '2026-02-21T11:57:59.709405+0000', 'last_undegraded': '2026-02-21T11:57:55.697396+0000', 'last_fullsized': '2026-02-21T11:57:55.697329+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 44, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'last_clean_scrub_stamp': '2026-02-21T11:56:55.590393+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-22T15:41:24.897768+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000745954, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4], 'acting': [4], 'avail_no_missing': ['4'], 'object_location_counts': [{'shards': '4', 'objects': 1}], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2026-02-21T11:58:06.486 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T11:58:06.676 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:06.677 INFO:teuthology.orchestra.run.trial196.stdout:{"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noout flag(s) set","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2/4 objects degraded (50.000%), 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":82,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":1},"osdmap":{"epoch":59,"num_osds":10,"num_up_osds":10,"osd_up_since":1771675080,"num_in_osds":10,"osd_in_since":1771675006,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+undersized+degraded","count":2},{"state_name":"active+undersized","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":282345472,"bytes_avail":1073459478528,"bytes_total":1073741824000,"degraded_objects":2,"degraded_total":4,"degraded_ratio":0.5},"fsmap":{"epoch":1,"btime":"2026-02-21T11:56:44:569344+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":4,"modified":"2026-02-21T11:57:52.830949+0000","services":{}},"progress_events":{}} 2026-02-21T11:58:06.690 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2026-02-21T11:58:06.690 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:06.859 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:06.860 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:06.870 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2026-02-21T11:58:06.832776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276160,"kb_used_data":6560,"kb_used_omap":288,"kb_used_meta":269151,"kb_avail":1048299840,"statfs":{"total":1073741824000,"available":1073459036160,"internally_reserved":0,"allocated":6717440,"data_stored":3320582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":295644,"internal_metadata":275610916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000752"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865842+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:58:04.865842+0000","last_peered":"2026-02-21T11:58:04.865842+0000","last_clean":"2026-02-21T11:58:04.865842+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:58:04.865842+0000","last_undegraded":"2026-02-21T11:58:04.865842+0000","last_fullsized":"2026-02-21T11:58:04.865842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":58,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.320375+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:58:04.320375+0000","last_peered":"2026-02-21T11:58:04.320375+0000","last_clean":"2026-02-21T11:58:04.320375+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:58:04.320375+0000","last_undegraded":"2026-02-21T11:58:04.320375+0000","last_fullsized":"2026-02-21T11:58:04.320375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716973+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:58:04.716973+0000","last_peered":"2026-02-21T11:58:04.716973+0000","last_clean":"2026-02-21T11:58:04.716973+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:58:04.716973+0000","last_undegraded":"2026-02-21T11:58:04.716973+0000","last_fullsized":"2026-02-21T11:58:04.716973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":58,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.036590+0000","last_change":"2026-02-21T11:58:01.957980+0000","last_active":"2026-02-21T11:58:04.036590+0000","last_peered":"2026-02-21T11:58:04.036590+0000","last_clean":"2026-02-21T11:58:04.036590+0000","last_became_active":"2026-02-21T11:58:01.957804+0000","last_became_peered":"2026-02-21T11:58:01.957804+0000","last_unstale":"2026-02-21T11:58:04.036590+0000","last_undegraded":"2026-02-21T11:58:04.036590+0000","last_fullsized":"2026-02-21T11:58:04.036590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:10:05.382284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865837+0000","last_change":"2026-02-21T11:58:01.715853+0000","last_active":"2026-02-21T11:58:04.865837+0000","last_peered":"2026-02-21T11:58:04.865837+0000","last_clean":"2026-02-21T11:58:04.865837+0000","last_became_active":"2026-02-21T11:58:01.715643+0000","last_became_peered":"2026-02-21T11:58:01.715643+0000","last_unstale":"2026-02-21T11:58:04.865837+0000","last_undegraded":"2026-02-21T11:58:04.865837+0000","last_fullsized":"2026-02-21T11:58:04.865837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:00:51.233910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716955+0000","last_change":"2026-02-21T11:58:01.715937+0000","last_active":"2026-02-21T11:58:04.716955+0000","last_peered":"2026-02-21T11:58:04.716955+0000","last_clean":"2026-02-21T11:58:04.716955+0000","last_became_active":"2026-02-21T11:58:01.715783+0000","last_became_peered":"2026-02-21T11:58:01.715783+0000","last_unstale":"2026-02-21T11:58:04.716955+0000","last_undegraded":"2026-02-21T11:58:04.716955+0000","last_fullsized":"2026-02-21T11:58:04.716955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:34:49.646250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:05.283832+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:58:05.283832+0000","last_peered":"2026-02-21T11:58:05.283832+0000","last_clean":"2026-02-21T11:58:05.283832+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:58:05.283832+0000","last_undegraded":"2026-02-21T11:58:05.283832+0000","last_fullsized":"2026-02-21T11:58:05.283832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.765653+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:58:04.765653+0000","last_peered":"2026-02-21T11:58:04.765653+0000","last_clean":"2026-02-21T11:58:04.765653+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:58:04.765653+0000","last_undegraded":"2026-02-21T11:58:04.765653+0000","last_fullsized":"2026-02-21T11:58:04.765653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30351,"internal_metadata":27560305},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593746,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29039,"internal_metadata":27561617},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593745,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29693,"internal_metadata":27560963},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28391,"internal_metadata":27562265},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":56,"seq":240518168593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27612,"kb_used_data":652,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829988,"statfs":{"total":107374182400,"available":107345907712,"internally_reserved":0,"allocated":667648,"data_stored":327223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:09.872 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:10.028 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:10.029 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:10.041 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2026-02-21T11:58:08.832993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276160,"kb_used_data":6560,"kb_used_omap":288,"kb_used_meta":269151,"kb_avail":1048299840,"statfs":{"total":1073741824000,"available":1073459036160,"internally_reserved":0,"allocated":6717440,"data_stored":3320582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":295644,"internal_metadata":275610916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.119237"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865842+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:58:04.865842+0000","last_peered":"2026-02-21T11:58:04.865842+0000","last_clean":"2026-02-21T11:58:04.865842+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:58:04.865842+0000","last_undegraded":"2026-02-21T11:58:04.865842+0000","last_fullsized":"2026-02-21T11:58:04.865842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":58,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.320375+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:58:04.320375+0000","last_peered":"2026-02-21T11:58:04.320375+0000","last_clean":"2026-02-21T11:58:04.320375+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:58:04.320375+0000","last_undegraded":"2026-02-21T11:58:04.320375+0000","last_fullsized":"2026-02-21T11:58:04.320375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716973+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:58:04.716973+0000","last_peered":"2026-02-21T11:58:04.716973+0000","last_clean":"2026-02-21T11:58:04.716973+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:58:04.716973+0000","last_undegraded":"2026-02-21T11:58:04.716973+0000","last_fullsized":"2026-02-21T11:58:04.716973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":58,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.036590+0000","last_change":"2026-02-21T11:58:01.957980+0000","last_active":"2026-02-21T11:58:04.036590+0000","last_peered":"2026-02-21T11:58:04.036590+0000","last_clean":"2026-02-21T11:58:04.036590+0000","last_became_active":"2026-02-21T11:58:01.957804+0000","last_became_peered":"2026-02-21T11:58:01.957804+0000","last_unstale":"2026-02-21T11:58:04.036590+0000","last_undegraded":"2026-02-21T11:58:04.036590+0000","last_fullsized":"2026-02-21T11:58:04.036590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:10:05.382284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865837+0000","last_change":"2026-02-21T11:58:01.715853+0000","last_active":"2026-02-21T11:58:04.865837+0000","last_peered":"2026-02-21T11:58:04.865837+0000","last_clean":"2026-02-21T11:58:04.865837+0000","last_became_active":"2026-02-21T11:58:01.715643+0000","last_became_peered":"2026-02-21T11:58:01.715643+0000","last_unstale":"2026-02-21T11:58:04.865837+0000","last_undegraded":"2026-02-21T11:58:04.865837+0000","last_fullsized":"2026-02-21T11:58:04.865837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:00:51.233910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716955+0000","last_change":"2026-02-21T11:58:01.715937+0000","last_active":"2026-02-21T11:58:04.716955+0000","last_peered":"2026-02-21T11:58:04.716955+0000","last_clean":"2026-02-21T11:58:04.716955+0000","last_became_active":"2026-02-21T11:58:01.715783+0000","last_became_peered":"2026-02-21T11:58:01.715783+0000","last_unstale":"2026-02-21T11:58:04.716955+0000","last_undegraded":"2026-02-21T11:58:04.716955+0000","last_fullsized":"2026-02-21T11:58:04.716955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:34:49.646250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:05.283832+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:58:05.283832+0000","last_peered":"2026-02-21T11:58:05.283832+0000","last_clean":"2026-02-21T11:58:05.283832+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:58:05.283832+0000","last_undegraded":"2026-02-21T11:58:05.283832+0000","last_fullsized":"2026-02-21T11:58:05.283832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.765653+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:58:04.765653+0000","last_peered":"2026-02-21T11:58:04.765653+0000","last_clean":"2026-02-21T11:58:04.765653+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:58:04.765653+0000","last_undegraded":"2026-02-21T11:58:04.765653+0000","last_fullsized":"2026-02-21T11:58:04.765653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30351,"internal_metadata":27560305},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593746,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29039,"internal_metadata":27561617},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593745,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29693,"internal_metadata":27560963},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28391,"internal_metadata":27562265},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":56,"seq":240518168593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27612,"kb_used_data":652,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829988,"statfs":{"total":107374182400,"available":107345907712,"internally_reserved":0,"allocated":667648,"data_stored":327223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:10.043 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T11:58:10.043 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:10.201 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:10.202 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:10.214 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2026-02-21T11:58:08.832993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276160,"kb_used_data":6560,"kb_used_omap":288,"kb_used_meta":269151,"kb_avail":1048299840,"statfs":{"total":1073741824000,"available":1073459036160,"internally_reserved":0,"allocated":6717440,"data_stored":3320582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":295644,"internal_metadata":275610916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.119237"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865842+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:58:04.865842+0000","last_peered":"2026-02-21T11:58:04.865842+0000","last_clean":"2026-02-21T11:58:04.865842+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:58:04.865842+0000","last_undegraded":"2026-02-21T11:58:04.865842+0000","last_fullsized":"2026-02-21T11:58:04.865842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":58,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.320375+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:58:04.320375+0000","last_peered":"2026-02-21T11:58:04.320375+0000","last_clean":"2026-02-21T11:58:04.320375+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:58:04.320375+0000","last_undegraded":"2026-02-21T11:58:04.320375+0000","last_fullsized":"2026-02-21T11:58:04.320375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716973+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:58:04.716973+0000","last_peered":"2026-02-21T11:58:04.716973+0000","last_clean":"2026-02-21T11:58:04.716973+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:58:04.716973+0000","last_undegraded":"2026-02-21T11:58:04.716973+0000","last_fullsized":"2026-02-21T11:58:04.716973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":81,"reported_epoch":58,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.036590+0000","last_change":"2026-02-21T11:58:01.957980+0000","last_active":"2026-02-21T11:58:04.036590+0000","last_peered":"2026-02-21T11:58:04.036590+0000","last_clean":"2026-02-21T11:58:04.036590+0000","last_became_active":"2026-02-21T11:58:01.957804+0000","last_became_peered":"2026-02-21T11:58:01.957804+0000","last_unstale":"2026-02-21T11:58:04.036590+0000","last_undegraded":"2026-02-21T11:58:04.036590+0000","last_fullsized":"2026-02-21T11:58:04.036590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:10:05.382284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865837+0000","last_change":"2026-02-21T11:58:01.715853+0000","last_active":"2026-02-21T11:58:04.865837+0000","last_peered":"2026-02-21T11:58:04.865837+0000","last_clean":"2026-02-21T11:58:04.865837+0000","last_became_active":"2026-02-21T11:58:01.715643+0000","last_became_peered":"2026-02-21T11:58:01.715643+0000","last_unstale":"2026-02-21T11:58:04.865837+0000","last_undegraded":"2026-02-21T11:58:04.865837+0000","last_fullsized":"2026-02-21T11:58:04.865837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:00:51.233910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716955+0000","last_change":"2026-02-21T11:58:01.715937+0000","last_active":"2026-02-21T11:58:04.716955+0000","last_peered":"2026-02-21T11:58:04.716955+0000","last_clean":"2026-02-21T11:58:04.716955+0000","last_became_active":"2026-02-21T11:58:01.715783+0000","last_became_peered":"2026-02-21T11:58:01.715783+0000","last_unstale":"2026-02-21T11:58:04.716955+0000","last_undegraded":"2026-02-21T11:58:04.716955+0000","last_fullsized":"2026-02-21T11:58:04.716955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:34:49.646250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:05.283832+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:58:05.283832+0000","last_peered":"2026-02-21T11:58:05.283832+0000","last_clean":"2026-02-21T11:58:05.283832+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:58:05.283832+0000","last_undegraded":"2026-02-21T11:58:05.283832+0000","last_fullsized":"2026-02-21T11:58:05.283832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.765653+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:58:04.765653+0000","last_peered":"2026-02-21T11:58:04.765653+0000","last_clean":"2026-02-21T11:58:04.765653+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:58:04.765653+0000","last_undegraded":"2026-02-21T11:58:04.765653+0000","last_fullsized":"2026-02-21T11:58:04.765653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30351,"internal_metadata":27560305},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593746,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29039,"internal_metadata":27561617},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593745,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29693,"internal_metadata":27560963},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28391,"internal_metadata":27562265},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":327204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":27562263},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":56,"seq":240518168593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27612,"kb_used_data":652,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829988,"statfs":{"total":107374182400,"available":107345907712,"internally_reserved":0,"allocated":667648,"data_stored":327223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:10.215 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T11:58:10.373 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:10.374 INFO:teuthology.orchestra.run.trial196.stdout:{"epoch":59,"fsid":"9c5344fb-5d4c-42a6-924b-048ca08c7e50","created":"2026-02-21T11:56:44.569498+0000","modified":"2026-02-21T11:58:04.715619+0000","last_up_change":"2026-02-21T11:58:00.708454+0000","last_in_change":"2026-02-21T11:56:46.394335+0000","flags":"noout,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800064,"flags_set":["noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T11:56:54.964058+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.80000001192092896,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1.0000001192092896,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23153a6f-a5f3-44a2-aacd-3d049090cd7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":52,"up_from":53,"up_thru":36,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6802","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6803","nonce":1308491399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6830","nonce":1311491399},{"type":"v1","addr":"10.20.193.196:6832","nonce":1311491399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6808","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6809","nonce":1308491399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6806","nonce":1308491399},{"type":"v1","addr":"10.20.193.196:6807","nonce":1308491399}]},"public_addr":"10.20.193.196:6803/1308491399","cluster_addr":"10.20.193.196:6832/1311491399","heartbeat_back_addr":"10.20.193.196:6809/1308491399","heartbeat_front_addr":"10.20.193.196:6807/1308491399","state":["exists","up"]},{"osd":1,"uuid":"a3359d61-4602-4a4e-8224-f7bd08750521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":55,"up_from":56,"up_thru":56,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6829","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6833","nonce":2003661087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6876","nonce":2006661087},{"type":"v1","addr":"10.20.193.196:6877","nonce":2006661087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6851","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6855","nonce":2003661087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6843","nonce":2003661087},{"type":"v1","addr":"10.20.193.196:6847","nonce":2003661087}]},"public_addr":"10.20.193.196:6833/2003661087","cluster_addr":"10.20.193.196:6877/2006661087","heartbeat_back_addr":"10.20.193.196:6855/2003661087","heartbeat_front_addr":"10.20.193.196:6847/2003661087","state":["exists","up"]},{"osd":2,"uuid":"3a9f2159-17a6-4b49-bf3c-1ada498cfa39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":0,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6874","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6875","nonce":1372045057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6814","nonce":1373045057},{"type":"v1","addr":"10.20.193.196:6816","nonce":1373045057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6880","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6881","nonce":1372045057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6878","nonce":1372045057},{"type":"v1","addr":"10.20.193.196:6879","nonce":1372045057}]},"public_addr":"10.20.193.196:6875/1372045057","cluster_addr":"10.20.193.196:6816/1373045057","heartbeat_back_addr":"10.20.193.196:6881/1372045057","heartbeat_front_addr":"10.20.193.196:6879/1372045057","state":["exists","up"]},{"osd":3,"uuid":"5533f509-0f80-4f4f-b3d1-7ec1b80d7f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":56,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6826","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6827","nonce":635434592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6882","nonce":636434592},{"type":"v1","addr":"10.20.193.196:6883","nonce":636434592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6838","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6841","nonce":635434592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6834","nonce":635434592},{"type":"v1","addr":"10.20.193.196:6837","nonce":635434592}]},"public_addr":"10.20.193.196:6827/635434592","cluster_addr":"10.20.193.196:6883/636434592","heartbeat_back_addr":"10.20.193.196:6841/635434592","heartbeat_front_addr":"10.20.193.196:6837/635434592","state":["exists","up"]},{"osd":4,"uuid":"ac04a66c-f8db-4ada-8bb7-974605771fee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":56,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6810","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6812","nonce":627255440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6836","nonce":628255440},{"type":"v1","addr":"10.20.193.196:6840","nonce":628255440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6821","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6823","nonce":627255440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6817","nonce":627255440},{"type":"v1","addr":"10.20.193.196:6819","nonce":627255440}]},"public_addr":"10.20.193.196:6812/627255440","cluster_addr":"10.20.193.196:6840/628255440","heartbeat_back_addr":"10.20.193.196:6823/627255440","heartbeat_front_addr":"10.20.193.196:6819/627255440","state":["exists","up"]},{"osd":5,"uuid":"b8f758d2-6548-44b8-ab83-55138eeb765e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":43,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6844","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6846","nonce":1440695253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6804","nonce":1441695253},{"type":"v1","addr":"10.20.193.196:6805","nonce":1441695253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6856","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6857","nonce":1440695253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6852","nonce":1440695253},{"type":"v1","addr":"10.20.193.196:6854","nonce":1440695253}]},"public_addr":"10.20.193.196:6846/1440695253","cluster_addr":"10.20.193.196:6805/1441695253","heartbeat_back_addr":"10.20.193.196:6857/1440695253","heartbeat_front_addr":"10.20.193.196:6854/1440695253","state":["exists","up"]},{"osd":6,"uuid":"6f0ba2be-32a4-4a70-a952-2980d9bf9801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":50,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6811","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6813","nonce":2180386325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6848","nonce":2181386325},{"type":"v1","addr":"10.20.193.196:6850","nonce":2181386325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6824","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6825","nonce":2180386325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6820","nonce":2180386325},{"type":"v1","addr":"10.20.193.196:6822","nonce":2180386325}]},"public_addr":"10.20.193.196:6813/2180386325","cluster_addr":"10.20.193.196:6850/2181386325","heartbeat_back_addr":"10.20.193.196:6825/2180386325","heartbeat_front_addr":"10.20.193.196:6822/2180386325","state":["exists","up"]},{"osd":7,"uuid":"f3ae4755-f066-4372-ae0c-f364ea536d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":43,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6828","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6831","nonce":1122662840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6815","nonce":1123662840},{"type":"v1","addr":"10.20.193.196:6818","nonce":1123662840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6849","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6853","nonce":1122662840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6842","nonce":1122662840},{"type":"v1","addr":"10.20.193.196:6845","nonce":1122662840}]},"public_addr":"10.20.193.196:6831/1122662840","cluster_addr":"10.20.193.196:6818/1123662840","heartbeat_back_addr":"10.20.193.196:6853/1122662840","heartbeat_front_addr":"10.20.193.196:6845/1122662840","state":["exists","up"]},{"osd":8,"uuid":"05cc8f5d-d979-4eb7-93c2-5484939706e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":42,"up_from":43,"up_thru":16,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6866","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6867","nonce":2631587950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6835","nonce":2633587950},{"type":"v1","addr":"10.20.193.196:6839","nonce":2633587950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6872","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6873","nonce":2631587950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6870","nonce":2631587950},{"type":"v1","addr":"10.20.193.196:6871","nonce":2631587950}]},"public_addr":"10.20.193.196:6867/2631587950","cluster_addr":"10.20.193.196:6839/2633587950","heartbeat_back_addr":"10.20.193.196:6873/2631587950","heartbeat_front_addr":"10.20.193.196:6871/2631587950","state":["exists","up"]},{"osd":9,"uuid":"e8713fc9-8526-41b8-91e8-573218e1b699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":36,"up_from":37,"up_thru":53,"down_at":33,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6858","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6859","nonce":1536643720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6868","nonce":1537643720},{"type":"v1","addr":"10.20.193.196:6869","nonce":1537643720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6864","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6865","nonce":1536643720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.196:6862","nonce":1536643720},{"type":"v1","addr":"10.20.193.196:6863","nonce":1536643720}]},"public_addr":"10.20.193.196:6859/1536643720","cluster_addr":"10.20.193.196:6869/1537643720","heartbeat_back_addr":"10.20.193.196:6865/1536643720","heartbeat_front_addr":"10.20.193.196:6863/1536643720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2026-02-21T11:57:54.690650+0000","laggy_probability":0.65700000524520874,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.742513+0000","dead_epoch":50},{"osd":1,"down_stamp":"2026-02-21T11:57:54.690650+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.851069+0000","dead_epoch":54},{"osd":2,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.843339+0000","dead_epoch":41},{"osd":3,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.869158+0000","dead_epoch":42},{"osd":4,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.768872+0000","dead_epoch":41},{"osd":5,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.876562+0000","dead_epoch":40},{"osd":6,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.854308+0000","dead_epoch":40},{"osd":7,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.901701+0000","dead_epoch":41},{"osd":8,"down_stamp":"2026-02-21T11:57:44.661454+0000","laggy_probability":0.50999999046325684,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.786611+0000","dead_epoch":41},{"osd":9,"down_stamp":"2026-02-21T11:57:37.645767+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T11:56:48.929061+0000","dead_epoch":34}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T11:58:11.387 INFO:tasks.ceph:Scrubbing osd.0 2026-02-21T11:58:11.387 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:11.466 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:11.466 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:11.466 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:11.476 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-21T11:58:11.629 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 0 to deep-scrub 2026-02-21T11:58:11.641 INFO:tasks.ceph:Scrubbing osd.1 2026-02-21T11:58:11.641 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:11.725 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:11.725 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:11.725 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:11.737 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-21T11:58:11.894 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 1 to deep-scrub 2026-02-21T11:58:11.905 INFO:tasks.ceph:Scrubbing osd.2 2026-02-21T11:58:11.905 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:11.983 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:11.983 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:11.983 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:11.992 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-21T11:58:12.153 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 2 to deep-scrub 2026-02-21T11:58:12.165 INFO:tasks.ceph:Scrubbing osd.3 2026-02-21T11:58:12.165 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:12.243 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:12.244 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:12.244 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:12.253 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-21T11:58:12.424 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 3 to deep-scrub 2026-02-21T11:58:12.436 INFO:tasks.ceph:Scrubbing osd.4 2026-02-21T11:58:12.436 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:12.512 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:12.512 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:12.512 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:12.521 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-21T11:58:12.678 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 4 to deep-scrub 2026-02-21T11:58:12.691 INFO:tasks.ceph:Scrubbing osd.5 2026-02-21T11:58:12.691 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:12.768 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:12.768 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:12.768 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:12.778 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-21T11:58:12.946 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 5 to deep-scrub 2026-02-21T11:58:12.957 INFO:tasks.ceph:Scrubbing osd.6 2026-02-21T11:58:12.958 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:13.042 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:13.042 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:13.043 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:13.052 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-21T11:58:13.217 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 6 to deep-scrub 2026-02-21T11:58:13.230 INFO:tasks.ceph:Scrubbing osd.7 2026-02-21T11:58:13.230 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:13.318 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:13.318 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:13.318 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:13.328 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-21T11:58:13.492 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 7 to deep-scrub 2026-02-21T11:58:13.505 INFO:tasks.ceph:Scrubbing osd.8 2026-02-21T11:58:13.505 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:13.584 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:13.584 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:13.584 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:13.593 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2026-02-21T11:58:13.758 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 8 to deep-scrub 2026-02-21T11:58:13.770 INFO:tasks.ceph:Scrubbing osd.9 2026-02-21T11:58:13.770 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2026-02-21T11:58:13.854 INFO:teuthology.orchestra.run.trial196.stdout:{ 2026-02-21T11:58:13.854 INFO:teuthology.orchestra.run.trial196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T11:58:13.855 INFO:teuthology.orchestra.run.trial196.stdout:} 2026-02-21T11:58:13.864 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2026-02-21T11:58:14.022 INFO:teuthology.orchestra.run.trial196.stderr:instructed osd(s) 9 to deep-scrub 2026-02-21T11:58:14.034 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:14.189 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:14.190 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:14.201 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2026-02-21T11:58:12.833413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276208,"kb_used_data":6608,"kb_used_omap":291,"kb_used_meta":269148,"kb_avail":1048299792,"statfs":{"total":1073741824000,"available":1073458987008,"internally_reserved":0,"allocated":6766592,"data_stored":3352918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":298248,"internal_metadata":275608312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.117477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865842+0000","last_change":"2026-02-21T11:57:48.679893+0000","last_active":"2026-02-21T11:58:04.865842+0000","last_peered":"2026-02-21T11:58:04.865842+0000","last_clean":"2026-02-21T11:58:04.865842+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:58:04.865842+0000","last_undegraded":"2026-02-21T11:58:04.865842+0000","last_fullsized":"2026-02-21T11:58:04.865842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:04:00.348524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:05.786017+0000","last_change":"2026-02-21T11:57:48.679921+0000","last_active":"2026-02-21T11:58:05.786017+0000","last_peered":"2026-02-21T11:58:05.786017+0000","last_clean":"2026-02-21T11:58:05.786017+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:58:05.786017+0000","last_undegraded":"2026-02-21T11:58:05.786017+0000","last_fullsized":"2026-02-21T11:58:05.786017+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:05:06.632805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716973+0000","last_change":"2026-02-21T11:57:58.971748+0000","last_active":"2026-02-21T11:58:04.716973+0000","last_peered":"2026-02-21T11:58:04.716973+0000","last_clean":"2026-02-21T11:58:04.716973+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:58:04.716973+0000","last_undegraded":"2026-02-21T11:58:04.716973+0000","last_fullsized":"2026-02-21T11:58:04.716973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:43:46.131329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:05.785935+0000","last_change":"2026-02-21T11:58:01.957980+0000","last_active":"2026-02-21T11:58:05.785935+0000","last_peered":"2026-02-21T11:58:05.785935+0000","last_clean":"2026-02-21T11:58:05.785935+0000","last_became_active":"2026-02-21T11:58:01.957804+0000","last_became_peered":"2026-02-21T11:58:01.957804+0000","last_unstale":"2026-02-21T11:58:05.785935+0000","last_undegraded":"2026-02-21T11:58:05.785935+0000","last_fullsized":"2026-02-21T11:58:05.785935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T14:10:05.382284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.865837+0000","last_change":"2026-02-21T11:58:01.715853+0000","last_active":"2026-02-21T11:58:04.865837+0000","last_peered":"2026-02-21T11:58:04.865837+0000","last_clean":"2026-02-21T11:58:04.865837+0000","last_became_active":"2026-02-21T11:58:01.715643+0000","last_became_peered":"2026-02-21T11:58:01.715643+0000","last_unstale":"2026-02-21T11:58:04.865837+0000","last_undegraded":"2026-02-21T11:58:04.865837+0000","last_fullsized":"2026-02-21T11:58:04.865837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:00:51.233910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.716955+0000","last_change":"2026-02-21T11:58:01.715937+0000","last_active":"2026-02-21T11:58:04.716955+0000","last_peered":"2026-02-21T11:58:04.716955+0000","last_clean":"2026-02-21T11:58:04.716955+0000","last_became_active":"2026-02-21T11:58:01.715783+0000","last_became_peered":"2026-02-21T11:58:01.715783+0000","last_unstale":"2026-02-21T11:58:04.716955+0000","last_undegraded":"2026-02-21T11:58:04.716955+0000","last_fullsized":"2026-02-21T11:58:04.716955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:34:49.646250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:05.283832+0000","last_change":"2026-02-21T11:57:48.679940+0000","last_active":"2026-02-21T11:58:05.283832+0000","last_peered":"2026-02-21T11:58:05.283832+0000","last_clean":"2026-02-21T11:58:05.283832+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:58:05.283832+0000","last_undegraded":"2026-02-21T11:58:05.283832+0000","last_fullsized":"2026-02-21T11:58:05.283832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:47:17.901717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:04.765653+0000","last_change":"2026-02-21T11:57:48.679807+0000","last_active":"2026-02-21T11:58:04.765653+0000","last_peered":"2026-02-21T11:58:04.765653+0000","last_clean":"2026-02-21T11:58:04.765653+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:58:04.765653+0000","last_undegraded":"2026-02-21T11:58:04.765653+0000","last_fullsized":"2026-02-21T11:58:04.765653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:56:55.590393+0000","last_clean_scrub_stamp":"2026-02-21T11:56:55.590393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:20:14.277611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30351,"internal_metadata":27560305},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593746,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593747,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29039,"internal_metadata":27561617},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593746,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30344,"internal_metadata":27560312},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593746,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593746,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593746,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28391,"internal_metadata":27562265},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593746,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":56,"seq":240518168594,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29695,"internal_metadata":27560961},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266707,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:14.207 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.207 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.207 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.208 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.208 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.208 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.208 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.209 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T11:56:55.590393+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=56, tm_sec=55, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=11, tm_min=58, tm_sec=10, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T11:58:14.209 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T11:58:34.210 DEBUG:teuthology.orchestra.run.trial196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T11:58:34.368 INFO:teuthology.orchestra.run.trial196.stdout: 2026-02-21T11:58:34.369 INFO:teuthology.orchestra.run.trial196.stderr:dumped all 2026-02-21T11:58:34.379 INFO:teuthology.orchestra.run.trial196.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2026-02-21T11:58:32.835550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276208,"kb_used_data":6608,"kb_used_omap":291,"kb_used_meta":269148,"kb_avail":1048299792,"statfs":{"total":1073741824000,"available":1073458987008,"internally_reserved":0,"allocated":6766592,"data_stored":3352918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":298248,"internal_metadata":275608312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001177"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:13.944865+0000","last_change":"2026-02-21T11:58:13.944865+0000","last_active":"2026-02-21T11:58:13.944865+0000","last_peered":"2026-02-21T11:58:13.944865+0000","last_clean":"2026-02-21T11:58:13.944865+0000","last_became_active":"2026-02-21T11:57:48.679580+0000","last_became_peered":"2026-02-21T11:57:48.679580+0000","last_unstale":"2026-02-21T11:58:13.944865+0000","last_undegraded":"2026-02-21T11:58:13.944865+0000","last_fullsized":"2026-02-21T11:58:13.944865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:58:13.944827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:58:13.944827+0000","last_clean_scrub_stamp":"2026-02-21T11:58:13.944827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T20:01:56.768891+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000205541,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:13.758087+0000","last_change":"2026-02-21T11:58:13.758087+0000","last_active":"2026-02-21T11:58:13.758087+0000","last_peered":"2026-02-21T11:58:13.758087+0000","last_clean":"2026-02-21T11:58:13.758087+0000","last_became_active":"2026-02-21T11:57:48.679724+0000","last_became_peered":"2026-02-21T11:57:48.679724+0000","last_unstale":"2026-02-21T11:58:13.758087+0000","last_undegraded":"2026-02-21T11:58:13.758087+0000","last_fullsized":"2026-02-21T11:58:13.758087+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:58:13.758041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:58:13.758041+0000","last_clean_scrub_stamp":"2026-02-21T11:58:13.758041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T21:54:51.592857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00018687700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:14.078865+0000","last_change":"2026-02-21T11:58:14.078865+0000","last_active":"2026-02-21T11:58:14.078865+0000","last_peered":"2026-02-21T11:58:14.078865+0000","last_clean":"2026-02-21T11:58:14.078865+0000","last_became_active":"2026-02-21T11:57:58.971605+0000","last_became_peered":"2026-02-21T11:57:58.971605+0000","last_unstale":"2026-02-21T11:58:14.078865+0000","last_undegraded":"2026-02-21T11:58:14.078865+0000","last_fullsized":"2026-02-21T11:58:14.078865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:58:14.078818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:58:14.078818+0000","last_clean_scrub_stamp":"2026-02-21T11:58:14.078818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T13:14:06.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022559899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:12.742973+0000","last_change":"2026-02-21T11:58:12.742973+0000","last_active":"2026-02-21T11:58:12.742973+0000","last_peered":"2026-02-21T11:58:12.742973+0000","last_clean":"2026-02-21T11:58:12.742973+0000","last_became_active":"2026-02-21T11:58:01.957804+0000","last_became_peered":"2026-02-21T11:58:01.957804+0000","last_unstale":"2026-02-21T11:58:12.742973+0000","last_undegraded":"2026-02-21T11:58:12.742973+0000","last_fullsized":"2026-02-21T11:58:12.742973+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:58:12.742927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:58:12.742927+0000","last_clean_scrub_stamp":"2026-02-21T11:58:12.742927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:51:47.946524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"59'2","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:12.945636+0000","last_change":"2026-02-21T11:58:12.945636+0000","last_active":"2026-02-21T11:58:12.945636+0000","last_peered":"2026-02-21T11:58:12.945636+0000","last_clean":"2026-02-21T11:58:12.945636+0000","last_became_active":"2026-02-21T11:58:01.715643+0000","last_became_peered":"2026-02-21T11:58:01.715643+0000","last_unstale":"2026-02-21T11:58:12.945636+0000","last_undegraded":"2026-02-21T11:58:12.945636+0000","last_fullsized":"2026-02-21T11:58:12.945636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2026-02-21T11:58:12.945567+0000","last_deep_scrub":"59'2","last_deep_scrub_stamp":"2026-02-21T11:58:12.945567+0000","last_clean_scrub_stamp":"2026-02-21T11:58:12.945567+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T12:21:16.747799+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"59'3","reported_seq":116,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:13.559130+0000","last_change":"2026-02-21T11:58:13.559130+0000","last_active":"2026-02-21T11:58:13.559130+0000","last_peered":"2026-02-21T11:58:13.559130+0000","last_clean":"2026-02-21T11:58:13.559130+0000","last_became_active":"2026-02-21T11:58:01.715783+0000","last_became_peered":"2026-02-21T11:58:01.715783+0000","last_unstale":"2026-02-21T11:58:13.559130+0000","last_undegraded":"2026-02-21T11:58:13.559130+0000","last_fullsized":"2026-02-21T11:58:13.559130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'3","last_scrub_stamp":"2026-02-21T11:58:13.559090+0000","last_deep_scrub":"59'3","last_deep_scrub_stamp":"2026-02-21T11:58:13.559090+0000","last_clean_scrub_stamp":"2026-02-21T11:58:13.559090+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:18:14.393287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074595399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:13.566921+0000","last_change":"2026-02-21T11:58:13.566921+0000","last_active":"2026-02-21T11:58:13.566921+0000","last_peered":"2026-02-21T11:58:13.566921+0000","last_clean":"2026-02-21T11:58:13.566921+0000","last_became_active":"2026-02-21T11:57:48.679402+0000","last_became_peered":"2026-02-21T11:57:48.679402+0000","last_unstale":"2026-02-21T11:58:13.566921+0000","last_undegraded":"2026-02-21T11:58:13.566921+0000","last_fullsized":"2026-02-21T11:58:13.566921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:58:13.566881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:58:13.566881+0000","last_clean_scrub_stamp":"2026-02-21T11:58:13.566881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T19:12:43.626997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000171659,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T11:58:13.707657+0000","last_change":"2026-02-21T11:58:13.707657+0000","last_active":"2026-02-21T11:58:13.707657+0000","last_peered":"2026-02-21T11:58:13.707657+0000","last_clean":"2026-02-21T11:58:13.707657+0000","last_became_active":"2026-02-21T11:57:48.679448+0000","last_became_peered":"2026-02-21T11:57:48.679448+0000","last_unstale":"2026-02-21T11:58:13.707657+0000","last_undegraded":"2026-02-21T11:58:13.707657+0000","last_fullsized":"2026-02-21T11:58:13.707657+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T11:58:13.707615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T11:58:13.707615+0000","last_clean_scrub_stamp":"2026-02-21T11:58:13.707615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T16:08:25.121049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30351,"internal_metadata":27560305},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":43,"seq":184683593750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":43,"seq":184683593751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29039,"internal_metadata":27561617},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593750,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30344,"internal_metadata":27560312},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":43,"seq":184683593750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":43,"seq":184683593750,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":43,"seq":184683593750,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28391,"internal_metadata":27562265},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":43,"seq":184683593750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":56,"seq":240518168598,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":335307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29695,"internal_metadata":27560961},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":53,"seq":227633266711,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":335288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T11:58:34.380 DEBUG:teuthology.orchestra.run.trial196:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T11:58:34.550 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T11:58:34.551 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T11:58:34.551 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T11:58:34.551 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:34.637 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T11:58:34.637 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T11:58:34.638 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:34.717 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T11:58:34.717 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T11:58:34.718 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:38.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-21T11:58:40.819 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T11:58:40.820 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-21T11:58:40.820 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:40.881 INFO:tasks.ceph.osd.3:Stopped 2026-02-21T11:58:40.882 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-21T11:58:40.882 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:40.972 INFO:tasks.ceph.osd.4:Stopped 2026-02-21T11:58:40.973 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-21T11:58:40.973 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.061 INFO:tasks.ceph.osd.5:Stopped 2026-02-21T11:58:41.061 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-21T11:58:41.061 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.120 INFO:tasks.ceph.osd.6:Stopped 2026-02-21T11:58:41.121 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-21T11:58:41.121 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.192 INFO:tasks.ceph.osd.7:Stopped 2026-02-21T11:58:41.193 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2026-02-21T11:58:41.193 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.260 INFO:tasks.ceph.osd.8:Stopped 2026-02-21T11:58:41.260 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2026-02-21T11:58:41.260 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.354 INFO:tasks.ceph.osd.9:Stopped 2026-02-21T11:58:41.354 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T11:58:41.354 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T11:58:41.354 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.386 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T11:58:41.387 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T11:58:41.387 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T11:58:41.387 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T11:58:41.441 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T11:58:41.441 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T11:58:41.441 DEBUG:teuthology.orchestra.run.trial196:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v 'but it is still running' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2026-02-21T11:58:41.498 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:58:41.499 DEBUG:teuthology.orchestra.run.trial196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T11:58:41.558 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:58:41.558 DEBUG:teuthology.orchestra.run.trial196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T11:58:41.610 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:58:41.611 DEBUG:teuthology.orchestra.run.trial196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T11:58:41.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:58:41.663 DEBUG:teuthology.orchestra.run.trial196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-21T11:58:41.715 INFO:tasks.ceph:Archiving mon data... 2026-02-21T11:58:41.717 DEBUG:teuthology.misc:Transferring archived files from trial196:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613/data/mon.a.tgz 2026-02-21T11:58:41.718 DEBUG:teuthology.orchestra.run.trial196:> mktemp 2026-02-21T11:58:41.764 INFO:teuthology.orchestra.run.trial196.stdout:/tmp/tmp.XVbW6YDDkW 2026-02-21T11:58:41.765 DEBUG:teuthology.orchestra.run.trial196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.XVbW6YDDkW 2026-02-21T11:58:41.839 DEBUG:teuthology.orchestra.run.trial196:> sudo chmod 0666 /tmp/tmp.XVbW6YDDkW 2026-02-21T11:58:41.938 DEBUG:teuthology.orchestra.remote:trial196:/tmp/tmp.XVbW6YDDkW is 221KB 2026-02-21T11:58:41.994 DEBUG:teuthology.orchestra.run.trial196:> rm -fr /tmp/tmp.XVbW6YDDkW 2026-02-21T11:58:41.999 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T11:58:41.999 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T11:58:42.095 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T11:58:42.095 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T11:58:42.098 DEBUG:teuthology.misc:Transferring archived files from trial196:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613/remote/trial196/crash 2026-02-21T11:58:42.099 DEBUG:teuthology.orchestra.run.trial196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T11:58:42.144 INFO:tasks.ceph:Compressing logs... 2026-02-21T11:58:42.144 DEBUG:teuthology.orchestra.run.trial196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T11:58:42.196 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27703.log 2026-02-21T11:58:42.196 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28131.log 2026-02-21T11:58:42.197 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28651.log 2026-02-21T11:58:42.197 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27703.log.gz 2026-02-21T11:58:42.197 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2026-02-21T11:58:42.197 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28131.log.gz 2026-02-21T11:58:42.198 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25910.log 2026-02-21T11:58:42.198 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28651.log.gz 2026-02-21T11:58:42.198 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28287.log 2026-02-21T11:58:42.198 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2026-02-21T11:58:42.198 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25260.log 2026-02-21T11:58:42.198 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25910.log.gz 2026-02-21T11:58:42.199 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28287.log.gz 2026-02-21T11:58:42.199 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27947.log 2026-02-21T11:58:42.199 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20988.log 2026-02-21T11:58:42.199 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25260.log.gz 2026-02-21T11:58:42.199 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25939.log 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27947.log.gz 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20988.log.gz 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27973.log 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28025.log 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25939.log: /var/log/ceph/ceph-client.admin.27973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25939.log.gz 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27973.log.gz 2026-02-21T11:58:42.200 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26225.log 2026-02-21T11:58:42.201 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25396.log 2026-02-21T11:58:42.201 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28025.log.gz 2026-02-21T11:58:42.201 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25386.log 2026-02-21T11:58:42.201 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26225.log.gz 2026-02-21T11:58:42.201 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25396.log.gz 2026-02-21T11:58:42.201 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20279.log 2026-02-21T11:58:42.202 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27730.log 2026-02-21T11:58:42.202 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25386.log: /var/log/ceph/ceph-client.admin.20279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25386.log.gz 2026-02-21T11:58:42.202 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20279.log.gz 2026-02-21T11:58:42.202 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25215.log 2026-02-21T11:58:42.202 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26197.log 2026-02-21T11:58:42.202 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27730.log.gz 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26886.log 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25215.log.gz 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28627.log 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26197.log.gz 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26886.log.gz 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-21T11:58:42.203 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25942.log 2026-02-21T11:58:42.204 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28627.log.gz 2026-02-21T11:58:42.204 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28183.log 2026-02-21T11:58:42.204 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28443.log 2026-02-21T11:58:42.204 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25942.log.gz 2026-02-21T11:58:42.204 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27677.log 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28183.log.gz 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26430.log 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28443.log.gz 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27552.log 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27677.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27677.log.gz 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25391.log 2026-02-21T11:58:42.205 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26430.log.gz 2026-02-21T11:58:42.206 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27552.log.gz 2026-02-21T11:58:42.206 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-21T11:58:42.206 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26564.log 2026-02-21T11:58:42.206 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25391.log.gz 2026-02-21T11:58:42.206 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25907.log 2026-02-21T11:58:42.207 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27817.log 2026-02-21T11:58:42.207 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26564.log.gz 2026-02-21T11:58:42.207 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25190.log 2026-02-21T11:58:42.207 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25907.log.gz 2026-02-21T11:58:42.207 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27817.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T11:58:42.208 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27817.log.gz 2026-02-21T11:58:42.208 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28313.log 2026-02-21T11:58:42.208 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25190.log.gz 2026-02-21T11:58:42.208 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25397.log 2026-02-21T11:58:42.209 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20207.log 2026-02-21T11:58:42.209 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28313.log.gz 2026-02-21T11:58:42.209 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2026-02-21T11:58:42.209 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25397.log.gz 2026-02-21T11:58:42.209 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26536.log 2026-02-21T11:58:42.209 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20207.log.gz 2026-02-21T11:58:42.210 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20255.log 2026-02-21T11:58:42.210 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28600.log 2026-02-21T11:58:42.210 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26536.log.gz 2026-02-21T11:58:42.210 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20255.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-21T11:58:42.210 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20255.log.gz 2026-02-21T11:58:42.211 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20182.log 2026-02-21T11:58:42.211 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28600.log.gz 2026-02-21T11:58:42.211 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20158.log 2026-02-21T11:58:42.211 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20182.log.gz 2026-02-21T11:58:42.211 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27058.log 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26812.log 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20158.log.gz 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27651.log 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27058.log.gz 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26713.log 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26812.log.gz 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27651.log.gz 2026-02-21T11:58:42.212 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26193.log 2026-02-21T11:58:42.213 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25648.log 2026-02-21T11:58:42.213 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26713.log.gz 2026-02-21T11:58:42.213 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26910.log 2026-02-21T11:58:42.213 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26193.log.gz 2026-02-21T11:58:42.213 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20431.log 2026-02-21T11:58:42.214 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25648.log: /var/log/ceph/ceph-client.admin.26910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25648.log.gz 2026-02-21T11:58:42.214 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26910.log.gz 2026-02-21T11:58:42.214 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T11:58:42.214 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26196.log 2026-02-21T11:58:42.214 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20431.log.gz 2026-02-21T11:58:42.214 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20000.log 2026-02-21T11:58:42.215 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.26196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26196.log.gz 2026-02-21T11:58:42.215 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25647.log 2026-02-21T11:58:42.215 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25312.log 2026-02-21T11:58:42.215 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20000.log: /var/log/ceph/ceph-client.admin.25647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20000.log.gz 2026-02-21T11:58:42.215 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25647.log.gz 2026-02-21T11:58:42.215 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25645.log 2026-02-21T11:58:42.216 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25312.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2026-02-21T11:58:42.216 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25312.log.gz 2026-02-21T11:58:42.216 INFO:teuthology.orchestra.run.trial196.stderr:gzip/var/log/ceph/ceph-client.admin.25645.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27257.log 2026-02-21T11:58:42.216 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25645.log.gz 2026-02-21T11:58:42.216 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25338.log 2026-02-21T11:58:42.217 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28235.log 2026-02-21T11:58:42.217 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27257.log.gz 2026-02-21T11:58:42.217 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25338.log.gz 2026-02-21T11:58:42.217 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26195.log 2026-02-21T11:58:42.217 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25884.log 2026-02-21T11:58:42.218 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28235.log: /var/log/ceph/ceph-client.admin.26195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28235.log.gz 2026-02-21T11:58:42.218 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26195.log.gz 2026-02-21T11:58:42.218 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25399.log 2026-02-21T11:58:42.218 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20327.log 2026-02-21T11:58:42.218 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25884.log.gz 2026-02-21T11:58:42.219 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28105.log 2026-02-21T11:58:42.219 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25399.log.gz 2026-02-21T11:58:42.219 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25899.log 2026-02-21T11:58:42.219 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20327.log.gz 2026-02-21T11:58:42.220 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26189.log 2026-02-21T11:58:42.220 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28105.log.gz 2026-02-21T11:58:42.221 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25899.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25899.log.gz 2026-02-21T11:58:42.221 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26588.log 2026-02-21T11:58:42.221 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25639.log 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26189.log.gz 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27084.log 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26588.log: /var/log/ceph/ceph-client.admin.25639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26588.log.gz 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25639.log.gz 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28469.log 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25913.log 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27084.log.gz 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28469.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28469.log.gz 2026-02-21T11:58:42.222 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-21T11:58:42.223 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27008.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25913.log.gz 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27406.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20114.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27008.log.gz 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27406.log.gz 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28548.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27033.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20114.log.gz 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27921.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28548.log: /var/log/ceph/ceph-client.admin.27033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27033.log.gz 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28548.log.gz 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27159.log 2026-02-21T11:58:42.224 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25658.log 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27921.log.gz 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20231.log 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27159.log.gz 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25387.log 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25658.log.gz 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20231.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20231.log.gz 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25640.log 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25387.log.gz 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25385.log 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27282.log 2026-02-21T11:58:42.227 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25640.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25385.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26192.log 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27282.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25286.log 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26737.log 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26192.log: /var/log/ceph/ceph-client.admin.25286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26192.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25286.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28079.log 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27233.log 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26737.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28209.log 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28079.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27233.log.gz 2026-02-21T11:58:42.228 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27577.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28209.log.gz 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26787.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27577.log.gz 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28157.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27307.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26787.log: /var/log/ceph/ceph-client.admin.28157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26787.log.gz 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28157.log.gz 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26861.log 2026-02-21T11:58:42.230 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27307.log.gz 2026-02-21T11:58:42.231 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27602.log 2026-02-21T11:58:42.231 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27765.log 2026-02-21T11:58:42.231 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26861.log.gz 2026-02-21T11:58:42.231 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28574.log 2026-02-21T11:58:42.231 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27602.log.gz 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25660.log 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27765.log.gz 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27791.log 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28574.log.gz 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25660.log.gz 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28051.log 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27791.log.gz 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28365.log 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27999.log 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T11:58:42.232 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28051.log.gz 2026-02-21T11:58:42.233 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28365.log.gz 2026-02-21T11:58:42.233 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27332.log 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27999.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27999.log.gz 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26144.log 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27332.log.gz 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27209.log 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26144.log.gz 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26837.log 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27209.log.gz 2026-02-21T11:58:42.234 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26190.log 2026-02-21T11:58:42.235 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26837.log.gz 2026-02-21T11:58:42.235 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25389.log 2026-02-21T11:58:42.235 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25643.log 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26190.log.gz 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27503.log 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25389.log.gz 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27627.log 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25643.log.gz 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27134.log 2026-02-21T11:58:42.236 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27503.log.gz 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27455.log 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27627.log.gz 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28417.log 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27134.log.gz 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27455.log.gz 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20351.log 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27869.log 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28417.log.gz 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20351.log.gz 2026-02-21T11:58:42.237 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28339.log 2026-02-21T11:58:42.238 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27869.log.gz 2026-02-21T11:58:42.238 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27895.log 2026-02-21T11:58:42.238 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28339.log.gz 2026-02-21T11:58:42.238 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25642.log 2026-02-21T11:58:42.238 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27895.log.gz 2026-02-21T11:58:42.238 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28522.log 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26762.log 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25642.log: 0.0%/var/log/ceph/ceph-client.admin.28522.log: -- replaced with /var/log/ceph/ceph-client.admin.25642.log.gz 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28522.log.gz 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26404.log 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26762.log.gz 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26640.log 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26688.log 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26404.log.gz 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26640.log.gz 2026-02-21T11:58:42.239 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26958.log 2026-02-21T11:58:42.240 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26934.log 2026-02-21T11:58:42.240 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26688.log.gz 2026-02-21T11:58:42.240 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25659.log 2026-02-21T11:58:42.240 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26958.log.gz 2026-02-21T11:58:42.240 INFO:teuthology.orchestra.run.trial196.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26456.log 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26934.log.gz 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25908.log 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25659.log.gz 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25398.log 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26456.log.gz 2026-02-21T11:58:42.241 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25908.log.gz 2026-02-21T11:58:42.242 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25400.log 2026-02-21T11:58:42.242 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25240.log 2026-02-21T11:58:42.242 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25398.log.gz 2026-02-21T11:58:42.242 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T11:58:42.242 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25400.log.gz 2026-02-21T11:58:42.242 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25240.log.gz 2026-02-21T11:58:42.243 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20375.log 2026-02-21T11:58:42.243 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25909.log 2026-02-21T11:58:42.243 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25912.log 2026-02-21T11:58:42.243 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20375.log.gz 2026-02-21T11:58:42.244 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25909.log.gz 2026-02-21T11:58:42.244 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28261.log 2026-02-21T11:58:42.244 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.25912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27479.log 2026-02-21T11:58:42.244 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25912.log.gz 2026-02-21T11:58:42.244 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28261.log.gz 2026-02-21T11:58:42.244 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27431.log 2026-02-21T11:58:42.245 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26664.log 2026-02-21T11:58:42.245 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27479.log.gz 2026-02-21T11:58:42.245 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27184.log 2026-02-21T11:58:42.245 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27431.log.gz 2026-02-21T11:58:42.246 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20064.log 2026-02-21T11:58:42.246 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26664.log.gz 2026-02-21T11:58:42.246 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27184.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T11:58:42.246 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27184.log.gz 2026-02-21T11:58:42.246 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28391.log 2026-02-21T11:58:42.246 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20064.log.gz 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28496.log 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28391.log.gz 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27381.log 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26149.log 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.28496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28496.log.gz 2026-02-21T11:58:42.247 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27381.log.gz 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26983.log 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27527.log 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26149.log.gz 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.26983.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12059.log 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26983.log.gz 2026-02-21T11:58:42.248 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20303.log 2026-02-21T11:58:42.249 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27527.log.gz 2026-02-21T11:58:42.249 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph.tmp-client.admin.12059.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12059.log.gz 2026-02-21T11:58:42.249 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27109.log 2026-02-21T11:58:42.249 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.20303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27357.log 2026-02-21T11:58:42.249 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20303.log.gz 2026-02-21T11:58:42.250 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27109.log.gz 2026-02-21T11:58:42.250 INFO:teuthology.orchestra.run.trial196.stderr:/var/log/ceph/ceph-client.admin.27357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27357.log.gz 2026-02-21T11:58:42.252 INFO:teuthology.orchestra.run.trial196.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-21T11:58:42.253 INFO:teuthology.orchestra.run.trial196.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T11:58:42.253 INFO:teuthology.orchestra.run.trial196.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2026-02-21T11:58:42.265 INFO:teuthology.orchestra.run.trial196.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-21T11:58:42.275 INFO:teuthology.orchestra.run.trial196.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T11:58:42.277 INFO:teuthology.orchestra.run.trial196.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T11:58:42.310 INFO:teuthology.orchestra.run.trial196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T11:58:42.311 INFO:teuthology.orchestra.run.trial196.stderr: 2026-02-21T11:58:42.311 INFO:teuthology.orchestra.run.trial196.stderr:real 0m0.120s 2026-02-21T11:58:42.311 INFO:teuthology.orchestra.run.trial196.stderr:user 0m0.512s 2026-02-21T11:58:42.311 INFO:teuthology.orchestra.run.trial196.stderr:sys 0m0.067s 2026-02-21T11:58:42.312 INFO:tasks.ceph:Archiving logs... 2026-02-21T11:58:42.312 DEBUG:teuthology.misc:Transferring archived files from trial196:/var/log/ceph to /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613/remote/trial196/log 2026-02-21T11:58:42.313 DEBUG:teuthology.orchestra.run.trial196:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T11:58:42.517 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T11:58:42.535 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T11:58:42.535 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T11:58:42.625 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T11:58:42.637 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T11:58:42.637 DEBUG:teuthology.orchestra.run.trial196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T11:58:42.684 INFO:teuthology.orchestra.run.trial196.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T11:58:42.685 INFO:teuthology.orchestra.run.trial196.stdout:============================================================================== 2026-02-21T11:58:42.685 INFO:teuthology.orchestra.run.trial196.stdout: frangipani.org 128.2.1.22 3 u 37 64 7 30.428 +3.805 0.458 2026-02-21T11:58:42.685 INFO:teuthology.orchestra.run.trial196.stdout: ntp.sfo.icanbwe 32.146.78.153 2 u 34 64 7 68.559 +4.210 0.797 2026-02-21T11:58:42.685 INFO:teuthology.orchestra.run.trial196.stdout: 23.95.35.34 (23 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T11:58:42.685 INFO:teuthology.orchestra.run.trial196.stdout: 172-104-209-204 168.61.215.74 4 u 32 64 7 3.452 +1.918 0.400 2026-02-21T11:58:42.685 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T11:58:42.697 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T11:58:42.697 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T11:58:42.712 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T11:58:42.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T11:58:42.744 INFO:teuthology.task.internal:Duration was 312.897133 seconds 2026-02-21T11:58:42.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T11:58:42.758 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T11:58:42.758 DEBUG:teuthology.orchestra.run.trial196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T11:58:42.782 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T11:58:42.782 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial196.front.sepia.ceph.com 2026-02-21T11:58:42.782 DEBUG:teuthology.orchestra.run.trial196:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T11:58:42.831 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T11:58:42.831 DEBUG:teuthology.orchestra.run.trial196:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T11:58:43.096 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T11:58:43.096 DEBUG:teuthology.orchestra.run.trial196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T11:58:43.105 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T11:58:43.105 INFO:teuthology.orchestra.run.trial196.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T11:58:43.105 INFO:teuthology.orchestra.run.trial196.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T11:58:43.106 INFO:teuthology.orchestra.run.trial196.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T11:58:43.106 INFO:teuthology.orchestra.run.trial196.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T11:58:43.174 INFO:teuthology.orchestra.run.trial196.stderr: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T11:58:43.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T11:58:43.193 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T11:58:43.193 DEBUG:teuthology.orchestra.run.trial196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T11:58:43.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T11:58:43.236 DEBUG:teuthology.orchestra.run.trial196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:58:43.275 INFO:teuthology.orchestra.run.trial196.stdout:kernel.core_pattern = core 2026-02-21T11:58:43.283 DEBUG:teuthology.orchestra.run.trial196:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:58:43.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:58:43.327 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T11:58:43.339 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T11:58:43.340 DEBUG:teuthology.misc:Transferring archived files from trial196:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62613/remote/trial196 2026-02-21T11:58:43.340 DEBUG:teuthology.orchestra.run.trial196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T11:58:43.407 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T11:58:43.407 DEBUG:teuthology.orchestra.run.trial196:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T11:58:43.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T11:58:43.444 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T11:58:43.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T11:58:43.457 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T11:58:43.458 DEBUG:teuthology.orchestra.run.trial196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T11:58:43.476 INFO:teuthology.orchestra.run.trial196.stdout: 1179687 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 11:58 /home/ubuntu/cephtest 2026-02-21T11:58:43.478 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T11:58:43.495 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T11:58:43.545 INFO:teuthology.run:Summary data: description: rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} duration: 312.89713311195374 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-21T11:58:43.545 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T11:58:43.658 INFO:teuthology.run:pass