2026-02-22T23:17:20.825 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-22T23:17:20.840 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T23:17:20.919 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '64379' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: global: mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5907 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial068.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK7DJMO3UkcSK6rdHn+1TRWzJUhsIgwF2qrB516Dx0ZYDUgJUfKVHrKRzPUXq8OtDZc7XTnGzZDrjfxEKJ9N8nU= tasks: - install: null - ceph: pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - divergent_priors: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-22_19:35:40 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-22T23:17:20.919 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-22T23:17:20.920 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-22T23:17:20.920 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-22T23:17:20.921 INFO:teuthology.task.internal:Checking packages... 2026-02-22T23:17:20.921 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-22T23:17:20.921 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-22T23:17:20.921 INFO:teuthology.packaging:ref: None 2026-02-22T23:17:20.922 INFO:teuthology.packaging:tag: None 2026-02-22T23:17:20.922 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:17:20.922 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:17:20.922 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:17:21.037 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:17:21.038 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-22T23:17:21.050 INFO:teuthology.task.internal:no buildpackages task found 2026-02-22T23:17:21.050 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-22T23:17:21.064 INFO:teuthology.task.internal:Saving configuration 2026-02-22T23:17:21.074 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-22T23:17:21.086 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-22T23:17:21.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial068.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 23:13:50.202886', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPX6LNk+gnGHILJXTfIB3TZ061IVSDQU2RHyAwUTJD8E'} 2026-02-22T23:17:21.163 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-22T23:17:21.175 INFO:teuthology.task.internal:roles: ubuntu@trial068.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-22T23:17:21.175 INFO:teuthology.run_tasks:Running task console_log... 2026-02-22T23:17:21.260 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7631ca2ffba0>, signals=[15]) 2026-02-22T23:17:21.260 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-22T23:17:21.271 INFO:teuthology.task.internal:Opening connections... 2026-02-22T23:17:21.271 DEBUG:teuthology.task.internal:connecting to ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:17:21.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:17:21.333 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-22T23:17:21.345 DEBUG:teuthology.orchestra.run.trial068:> uname -m 2026-02-22T23:17:21.410 INFO:teuthology.orchestra.run.trial068.stdout:x86_64 2026-02-22T23:17:21.411 DEBUG:teuthology.orchestra.run.trial068:> cat /etc/os-release 2026-02-22T23:17:21.458 INFO:teuthology.orchestra.run.trial068.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:NAME="Ubuntu" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:VERSION_ID="22.04" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:VERSION_CODENAME=jammy 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:ID=ubuntu 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:ID_LIKE=debian 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-22T23:17:21.459 INFO:teuthology.orchestra.run.trial068.stdout:UBUNTU_CODENAME=jammy 2026-02-22T23:17:21.460 INFO:teuthology.lock.ops:Updating trial068.front.sepia.ceph.com on lock server 2026-02-22T23:17:21.548 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-22T23:17:21.559 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-22T23:17:21.570 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-22T23:17:21.571 DEBUG:teuthology.orchestra.run.trial068:> test '!' -e /home/ubuntu/cephtest 2026-02-22T23:17:21.574 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-22T23:17:21.584 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-22T23:17:21.585 DEBUG:teuthology.orchestra.run.trial068:> test -z $(ls -A /var/lib/ceph) 2026-02-22T23:17:21.619 INFO:teuthology.orchestra.run.trial068.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-22T23:17:21.620 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-22T23:17:21.711 INFO:teuthology.run_tasks:Running task kernel... 2026-02-22T23:17:21.731 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-22T23:17:21.731 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-22T23:17:21.731 DEBUG:teuthology.orchestra.run.trial068:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T23:17:21.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:17:21.734 DEBUG:teuthology.orchestra.run.trial068:> uname -r 2026-02-22T23:17:21.779 INFO:teuthology.orchestra.run.trial068.stdout:5.15.0-170-generic 2026-02-22T23:17:21.779 INFO:teuthology.task.kernel:Running kernel on trial068: 5.15.0-170-generic 2026-02-22T23:17:21.779 DEBUG:teuthology.orchestra.run.trial068:> sudo apt-get clean 2026-02-22T23:17:21.837 DEBUG:teuthology.orchestra.run.trial068:> sudo apt-get update 2026-02-22T23:17:22.287 INFO:teuthology.orchestra.run.trial068.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-22T23:17:22.360 INFO:teuthology.orchestra.run.trial068.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-22T23:17:22.636 INFO:teuthology.orchestra.run.trial068.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-22T23:17:22.757 INFO:teuthology.orchestra.run.trial068.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-22T23:17:22.871 INFO:teuthology.orchestra.run.trial068.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-22T23:17:23.200 INFO:teuthology.orchestra.run.trial068.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-22T23:17:23.209 INFO:teuthology.orchestra.run.trial068.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-22T23:17:23.209 INFO:teuthology.orchestra.run.trial068.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-22T23:17:23.238 INFO:teuthology.orchestra.run.trial068.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-22T23:17:23.244 INFO:teuthology.orchestra.run.trial068.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-22T23:17:23.244 INFO:teuthology.orchestra.run.trial068.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-22T23:17:23.373 INFO:teuthology.orchestra.run.trial068.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-22T23:17:23.391 INFO:teuthology.orchestra.run.trial068.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-22T23:17:23.462 INFO:teuthology.orchestra.run.trial068.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-22T23:17:23.469 INFO:teuthology.orchestra.run.trial068.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-22T23:17:23.470 INFO:teuthology.orchestra.run.trial068.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-22T23:17:23.498 INFO:teuthology.orchestra.run.trial068.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-22T23:17:23.499 INFO:teuthology.orchestra.run.trial068.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-22T23:17:23.620 INFO:teuthology.orchestra.run.trial068.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-22T23:17:27.181 INFO:teuthology.orchestra.run.trial068.stdout:Fetched 22.6 MB in 3s (8547 kB/s) 2026-02-22T23:17:27.704 INFO:teuthology.orchestra.run.trial068.stdout:Reading package lists... 2026-02-22T23:17:27.718 DEBUG:teuthology.orchestra.run.trial068:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-22T23:17:27.751 INFO:teuthology.orchestra.run.trial068.stdout:Reading package lists... 2026-02-22T23:17:27.857 INFO:teuthology.orchestra.run.trial068.stdout:Building dependency tree... 2026-02-22T23:17:27.857 INFO:teuthology.orchestra.run.trial068.stdout:Reading state information... 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout:linux-image-generic set to manually installed. 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout:The following packages were automatically installed and are no longer required: 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout: sg3-utils-udev 2026-02-22T23:17:27.944 INFO:teuthology.orchestra.run.trial068.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-22T23:17:27.963 INFO:teuthology.orchestra.run.trial068.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-22T23:17:27.965 DEBUG:teuthology.orchestra.run.trial068:> dpkg -s linux-image-generic 2026-02-22T23:17:28.017 INFO:teuthology.orchestra.run.trial068.stdout:Package: linux-image-generic 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Status: install ok installed 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Priority: optional 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Section: kernel 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Installed-Size: 21 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Maintainer: Ubuntu Kernel Team 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Architecture: amd64 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Source: linux-meta 2026-02-22T23:17:28.018 INFO:teuthology.orchestra.run.trial068.stdout:Version: 5.15.0.170.159 2026-02-22T23:17:28.019 INFO:teuthology.orchestra.run.trial068.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-22T23:17:28.019 INFO:teuthology.orchestra.run.trial068.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-22T23:17:28.019 INFO:teuthology.orchestra.run.trial068.stdout:Recommends: thermald 2026-02-22T23:17:28.019 INFO:teuthology.orchestra.run.trial068.stdout:Description: Generic Linux kernel image 2026-02-22T23:17:28.019 INFO:teuthology.orchestra.run.trial068.stdout: This package will always depend on the latest generic kernel image 2026-02-22T23:17:28.019 INFO:teuthology.orchestra.run.trial068.stdout: available. 2026-02-22T23:17:28.019 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T23:17:28.020 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-22T23:17:28.020 DEBUG:teuthology.orchestra.run.trial068:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T23:17:28.067 INFO:teuthology.orchestra.run.trial068.stdout:ttyS1 2026-02-22T23:17:28.094 DEBUG:teuthology.parallel:result is None 2026-02-22T23:17:28.094 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-22T23:17:28.107 INFO:teuthology.task.internal:Creating test directory... 2026-02-22T23:17:28.108 DEBUG:teuthology.orchestra.run.trial068:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T23:17:28.143 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-22T23:17:28.155 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-22T23:17:28.169 INFO:teuthology.task.internal:Creating archive directory... 2026-02-22T23:17:28.169 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T23:17:28.192 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-22T23:17:28.204 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-22T23:17:28.204 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T23:17:28.240 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:17:28.246 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:17:28.248 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-22T23:17:28.260 INFO:teuthology.task.internal:Configuring sudo... 2026-02-22T23:17:28.260 DEBUG:teuthology.orchestra.run.trial068:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T23:17:28.300 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-22T23:17:28.317 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-22T23:17:28.318 DEBUG:teuthology.orchestra.run.trial068:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T23:17:28.347 DEBUG:teuthology.orchestra.run.trial068:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:17:28.391 DEBUG:teuthology.orchestra.run.trial068:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:17:28.435 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:17:28.436 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T23:17:28.484 DEBUG:teuthology.orchestra.run.trial068:> sudo service rsyslog restart 2026-02-22T23:17:28.545 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-22T23:17:28.557 INFO:teuthology.task.internal:Starting timer... 2026-02-22T23:17:28.558 INFO:teuthology.run_tasks:Running task pcp... 2026-02-22T23:17:28.580 INFO:teuthology.run_tasks:Running task selinux... 2026-02-22T23:17:28.597 DEBUG:teuthology.task.selinux:Excluding trial068: OS 'ubuntu' does not support SELinux 2026-02-22T23:17:28.597 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-22T23:17:28.597 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-22T23:17:28.597 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-22T23:17:28.597 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-22T23:17:28.615 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-22T23:17:28.628 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-22T23:17:28.629 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial068.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-22T23:19:51.631 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial068.front.sepia.ceph.com')] 2026-02-22T23:19:51.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-22T23:19:51.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:19:51.743 DEBUG:teuthology.orchestra.run.trial068:> true 2026-02-22T23:19:51.991 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-22T23:19:51.991 INFO:teuthology.run_tasks:Running task clock... 2026-02-22T23:19:52.011 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-22T23:19:52.011 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T23:19:52.012 DEBUG:teuthology.orchestra.run.trial068:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:19:52.052 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-22T23:19:52.052 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Command line: ntpd -gq 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: ---------------------------------------------------- 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: ntp-4 is maintained by Network Time Foundation, 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: corporation. Support and training for ntp-4 are 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: available at https://www.nwtime.org/support 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: ---------------------------------------------------- 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: proto: precision = 0.020 usec (-25) 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: basedate set to 2022-02-04 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: gps base set to 2022-02-06 (week 2196) 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-22T23:19:52.054 INFO:teuthology.orchestra.run.trial068.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-22T23:19:52.054 INFO:teuthology.orchestra.run.trial068.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-22T23:19:52.055 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: restrict ::: KOD does nothing without LIMITED. 2026-02-22T23:19:52.055 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listen and drop on 0 v6wildcard [::]:123 2026-02-22T23:19:52.055 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-22T23:19:52.055 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listen normally on 2 lo 127.0.0.1:123 2026-02-22T23:19:52.055 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listen normally on 3 enp1s0f0 10.20.193.68:123 2026-02-22T23:19:52.055 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listen normally on 4 lo [::1]:123 2026-02-22T23:19:52.056 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:60ea%2]:123 2026-02-22T23:19:52.056 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:52 ntpd[9422]: Listening on routing socket on fd #22 for interface updates 2026-02-22T23:19:53.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:53.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:53.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:53.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:53.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:53.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:53.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:53.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:53.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:53.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:53.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:53.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:53 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:54.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:54 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:54.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:54 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:54.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:54 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:54.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:54 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:54.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:54 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:54.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:54 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:55.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:55.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:55.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:55.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:55.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:55.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:55.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:55.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:55.081 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:55.081 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:55.081 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:55.081 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:55 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:56.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:56 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:56.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:56 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:56.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:56 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:56.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:56 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:56.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:56 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:56.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:56 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:57.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:57.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:57.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:57.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:57.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:57.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:57.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:57.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:57.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:57.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:57.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:57.082 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:57.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:57.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:57.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:57.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:57.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:57.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:57 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:58.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:58 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:58.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:58 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:58.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:58 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:58.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:58 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:58.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:58 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:58.083 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:58 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:59.052 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:59.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:59.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:59.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:59.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:59.053 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:59.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:19:59.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:19:59.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:19:59.074 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:19:59.074 INFO:teuthology.orchestra.run.trial068.stdout:22 Feb 23:19:59 ntpd[9422]: ntpd: time set -0.122131 s 2026-02-22T23:19:59.074 INFO:teuthology.orchestra.run.trial068.stdout:ntpd: time set -0.122131s 2026-02-22T23:19:59.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-22T23:19:59.075 INFO:teuthology.orchestra.run.trial068.stderr:22 Feb 23:19:59 ntpd[9422]: can't open /var/log/ntpstats/loopstats.20260222: Permission denied 2026-02-22T23:19:59.142 INFO:teuthology.orchestra.run.trial068.stdout: remote refid st t when poll reach delay offset jitter 2026-02-22T23:19:59.142 INFO:teuthology.orchestra.run.trial068.stdout:============================================================================== 2026-02-22T23:19:59.142 INFO:teuthology.orchestra.run.trial068.stdout: 73.185.182.209 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-22T23:19:59.142 INFO:teuthology.orchestra.run.trial068.stdout: 152.67.232.7 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-22T23:19:59.142 INFO:teuthology.orchestra.run.trial068.stdout: nyc2.us.ntp.li .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-22T23:19:59.142 INFO:teuthology.orchestra.run.trial068.stdout: ntp.maxhost.io .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-22T23:19:59.143 INFO:teuthology.run_tasks:Running task install... 2026-02-22T23:19:59.155 DEBUG:teuthology.task.install:project ceph 2026-02-22T23:19:59.155 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}} 2026-02-22T23:19:59.155 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'} 2026-02-22T23:19:59.155 INFO:teuthology.task.install:Using flavor: default 2026-02-22T23:19:59.164 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-22T23:19:59.164 INFO:teuthology.task.install:extra packages: [] 2026-02-22T23:19:59.165 DEBUG:teuthology.orchestra.run.trial068:> sudo apt-key list | grep Ceph 2026-02-22T23:19:59.236 INFO:teuthology.orchestra.run.trial068.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-22T23:19:59.263 INFO:teuthology.orchestra.run.trial068.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-22T23:19:59.263 INFO:teuthology.orchestra.run.trial068.stdout:uid [ unknown] Ceph.com (release key) 2026-02-22T23:19:59.265 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-22T23:19:59.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:19:59.436 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default/ 2026-02-22T23:19:59.436 INFO:teuthology.task.install.deb:Package version is 19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:19:59.594 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:19:59.594 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-22T23:19:59.603 DEBUG:teuthology.orchestra.run.trial068:> sudo apt-get update 2026-02-22T23:19:59.780 INFO:teuthology.orchestra.run.trial068.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy InRelease 2026-02-22T23:19:59.792 INFO:teuthology.orchestra.run.trial068.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy Release [7662 B] 2026-02-22T23:19:59.804 INFO:teuthology.orchestra.run.trial068.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-22T23:19:59.816 INFO:teuthology.orchestra.run.trial068.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-22T23:20:00.034 INFO:teuthology.orchestra.run.trial068.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-22T23:20:00.100 INFO:teuthology.orchestra.run.trial068.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-22T23:20:00.172 INFO:teuthology.orchestra.run.trial068.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-22T23:20:00.244 INFO:teuthology.orchestra.run.trial068.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-22T23:20:00.389 INFO:teuthology.orchestra.run.trial068.stdout:Fetched 25.8 kB in 1s (41.1 kB/s) 2026-02-22T23:20:00.915 INFO:teuthology.orchestra.run.trial068.stdout:Reading package lists... 2026-02-22T23:20:00.928 DEBUG:teuthology.orchestra.run.trial068:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-689-gf90f4eae-1jammy cephadm=19.2.3-689-gf90f4eae-1jammy ceph-mds=19.2.3-689-gf90f4eae-1jammy ceph-mgr=19.2.3-689-gf90f4eae-1jammy ceph-common=19.2.3-689-gf90f4eae-1jammy ceph-fuse=19.2.3-689-gf90f4eae-1jammy ceph-test=19.2.3-689-gf90f4eae-1jammy ceph-volume=19.2.3-689-gf90f4eae-1jammy radosgw=19.2.3-689-gf90f4eae-1jammy python3-rados=19.2.3-689-gf90f4eae-1jammy python3-rgw=19.2.3-689-gf90f4eae-1jammy python3-cephfs=19.2.3-689-gf90f4eae-1jammy python3-rbd=19.2.3-689-gf90f4eae-1jammy libcephfs2=19.2.3-689-gf90f4eae-1jammy libcephfs-dev=19.2.3-689-gf90f4eae-1jammy librados2=19.2.3-689-gf90f4eae-1jammy librbd1=19.2.3-689-gf90f4eae-1jammy rbd-fuse=19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:20:00.979 INFO:teuthology.orchestra.run.trial068.stdout:Reading package lists... 2026-02-22T23:20:01.084 INFO:teuthology.orchestra.run.trial068.stdout:Building dependency tree... 2026-02-22T23:20:01.085 INFO:teuthology.orchestra.run.trial068.stdout:Reading state information... 2026-02-22T23:20:01.175 INFO:teuthology.orchestra.run.trial068.stdout:The following packages were automatically installed and are no longer required: 2026-02-22T23:20:01.175 INFO:teuthology.orchestra.run.trial068.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: sg3-utils-udev 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout:The following additional packages will be installed: 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-22T23:20:01.176 INFO:teuthology.orchestra.run.trial068.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-22T23:20:01.177 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout:Suggested packages: 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-22T23:20:01.178 INFO:teuthology.orchestra.run.trial068.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-22T23:20:01.179 INFO:teuthology.orchestra.run.trial068.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-22T23:20:01.179 INFO:teuthology.orchestra.run.trial068.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-22T23:20:01.179 INFO:teuthology.orchestra.run.trial068.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-22T23:20:01.179 INFO:teuthology.orchestra.run.trial068.stdout:Recommended packages: 2026-02-22T23:20:01.179 INFO:teuthology.orchestra.run.trial068.stdout: btrfs-tools 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout:The following NEW packages will be installed: 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-22T23:20:01.209 INFO:teuthology.orchestra.run.trial068.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-22T23:20:01.210 INFO:teuthology.orchestra.run.trial068.stdout: xmlstarlet zip 2026-02-22T23:20:01.291 INFO:teuthology.orchestra.run.trial068.stdout:0 upgraded, 108 newly installed, 0 to remove and 13 not upgraded. 2026-02-22T23:20:01.291 INFO:teuthology.orchestra.run.trial068.stdout:Need to get 178 MB of archives. 2026-02-22T23:20:01.292 INFO:teuthology.orchestra.run.trial068.stdout:After this operation, 809 MB of additional disk space will be used. 2026-02-22T23:20:01.292 INFO:teuthology.orchestra.run.trial068.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-689-gf90f4eae-1jammy [3597 kB] 2026-02-22T23:20:01.394 INFO:teuthology.orchestra.run.trial068.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-689-gf90f4eae-1jammy [3256 kB] 2026-02-22T23:20:01.439 INFO:teuthology.orchestra.run.trial068.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-689-gf90f4eae-1jammy [979 kB] 2026-02-22T23:20:01.454 INFO:teuthology.orchestra.run.trial068.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-689-gf90f4eae-1jammy [357 kB] 2026-02-22T23:20:01.460 INFO:teuthology.orchestra.run.trial068.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-689-gf90f4eae-1jammy [32.9 kB] 2026-02-22T23:20:01.461 INFO:teuthology.orchestra.run.trial068.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-689-gf90f4eae-1jammy [184 kB] 2026-02-22T23:20:01.464 INFO:teuthology.orchestra.run.trial068.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-689-gf90f4eae-1jammy [70.0 kB] 2026-02-22T23:20:01.464 INFO:teuthology.orchestra.run.trial068.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-689-gf90f4eae-1jammy [335 kB] 2026-02-22T23:20:01.471 INFO:teuthology.orchestra.run.trial068.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-689-gf90f4eae-1jammy [6935 kB] 2026-02-22T23:20:01.585 INFO:teuthology.orchestra.run.trial068.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-689-gf90f4eae-1jammy [112 kB] 2026-02-22T23:20:01.586 INFO:teuthology.orchestra.run.trial068.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-689-gf90f4eae-1jammy [470 kB] 2026-02-22T23:20:01.593 INFO:teuthology.orchestra.run.trial068.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-689-gf90f4eae-1jammy [26.5 MB] 2026-02-22T23:20:01.623 INFO:teuthology.orchestra.run.trial068.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-22T23:20:02.032 INFO:teuthology.orchestra.run.trial068.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-689-gf90f4eae-1jammy [5178 kB] 2026-02-22T23:20:02.037 INFO:teuthology.orchestra.run.trial068.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-22T23:20:02.050 INFO:teuthology.orchestra.run.trial068.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-22T23:20:02.113 INFO:teuthology.orchestra.run.trial068.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-689-gf90f4eae-1jammy [248 kB] 2026-02-22T23:20:02.116 INFO:teuthology.orchestra.run.trial068.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-689-gf90f4eae-1jammy [125 kB] 2026-02-22T23:20:02.118 INFO:teuthology.orchestra.run.trial068.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-689-gf90f4eae-1jammy [1081 kB] 2026-02-22T23:20:02.135 INFO:teuthology.orchestra.run.trial068.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-22T23:20:02.136 INFO:teuthology.orchestra.run.trial068.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-689-gf90f4eae-1jammy [6239 kB] 2026-02-22T23:20:02.239 INFO:teuthology.orchestra.run.trial068.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-689-gf90f4eae-1jammy [23.0 MB] 2026-02-22T23:20:02.379 INFO:teuthology.orchestra.run.trial068.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-22T23:20:02.391 INFO:teuthology.orchestra.run.trial068.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-22T23:20:02.424 INFO:teuthology.orchestra.run.trial068.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-22T23:20:02.434 INFO:teuthology.orchestra.run.trial068.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-22T23:20:02.436 INFO:teuthology.orchestra.run.trial068.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-22T23:20:02.437 INFO:teuthology.orchestra.run.trial068.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-22T23:20:02.438 INFO:teuthology.orchestra.run.trial068.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-22T23:20:02.459 INFO:teuthology.orchestra.run.trial068.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-22T23:20:02.463 INFO:teuthology.orchestra.run.trial068.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-22T23:20:02.467 INFO:teuthology.orchestra.run.trial068.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-22T23:20:02.551 INFO:teuthology.orchestra.run.trial068.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-22T23:20:02.551 INFO:teuthology.orchestra.run.trial068.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-22T23:20:02.553 INFO:teuthology.orchestra.run.trial068.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-22T23:20:02.555 INFO:teuthology.orchestra.run.trial068.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-22T23:20:02.557 INFO:teuthology.orchestra.run.trial068.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-22T23:20:02.558 INFO:teuthology.orchestra.run.trial068.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-22T23:20:02.558 INFO:teuthology.orchestra.run.trial068.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-22T23:20:02.559 INFO:teuthology.orchestra.run.trial068.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-22T23:20:02.618 INFO:teuthology.orchestra.run.trial068.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-689-gf90f4eae-1jammy [14.2 kB] 2026-02-22T23:20:02.618 INFO:teuthology.orchestra.run.trial068.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-689-gf90f4eae-1jammy [1173 kB] 2026-02-22T23:20:02.634 INFO:teuthology.orchestra.run.trial068.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-689-gf90f4eae-1jammy [2503 kB] 2026-02-22T23:20:02.645 INFO:teuthology.orchestra.run.trial068.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-22T23:20:02.645 INFO:teuthology.orchestra.run.trial068.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-22T23:20:02.646 INFO:teuthology.orchestra.run.trial068.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-22T23:20:02.646 INFO:teuthology.orchestra.run.trial068.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-22T23:20:02.646 INFO:teuthology.orchestra.run.trial068.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-22T23:20:02.675 INFO:teuthology.orchestra.run.trial068.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-689-gf90f4eae-1jammy [798 kB] 2026-02-22T23:20:02.688 INFO:teuthology.orchestra.run.trial068.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-689-gf90f4eae-1jammy [157 kB] 2026-02-22T23:20:02.690 INFO:teuthology.orchestra.run.trial068.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-689-gf90f4eae-1jammy [2396 kB] 2026-02-22T23:20:02.730 INFO:teuthology.orchestra.run.trial068.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-689-gf90f4eae-1jammy [8625 kB] 2026-02-22T23:20:02.733 INFO:teuthology.orchestra.run.trial068.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-22T23:20:02.736 INFO:teuthology.orchestra.run.trial068.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-22T23:20:02.736 INFO:teuthology.orchestra.run.trial068.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-22T23:20:02.737 INFO:teuthology.orchestra.run.trial068.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-22T23:20:02.737 INFO:teuthology.orchestra.run.trial068.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-22T23:20:02.821 INFO:teuthology.orchestra.run.trial068.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-22T23:20:02.821 INFO:teuthology.orchestra.run.trial068.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-22T23:20:02.822 INFO:teuthology.orchestra.run.trial068.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-22T23:20:02.823 INFO:teuthology.orchestra.run.trial068.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-22T23:20:02.823 INFO:teuthology.orchestra.run.trial068.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-22T23:20:02.872 INFO:teuthology.orchestra.run.trial068.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-689-gf90f4eae-1jammy [14.3 kB] 2026-02-22T23:20:02.872 INFO:teuthology.orchestra.run.trial068.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-689-gf90f4eae-1jammy [52.1 MB] 2026-02-22T23:20:02.908 INFO:teuthology.orchestra.run.trial068.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-22T23:20:02.908 INFO:teuthology.orchestra.run.trial068.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-22T23:20:02.909 INFO:teuthology.orchestra.run.trial068.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-22T23:20:02.909 INFO:teuthology.orchestra.run.trial068.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-22T23:20:02.910 INFO:teuthology.orchestra.run.trial068.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-22T23:20:02.996 INFO:teuthology.orchestra.run.trial068.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-22T23:20:02.997 INFO:teuthology.orchestra.run.trial068.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-22T23:20:03.000 INFO:teuthology.orchestra.run.trial068.stdout:Get:72 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-22T23:20:03.000 INFO:teuthology.orchestra.run.trial068.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-22T23:20:03.001 INFO:teuthology.orchestra.run.trial068.stdout:Get:74 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-22T23:20:03.084 INFO:teuthology.orchestra.run.trial068.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-22T23:20:03.087 INFO:teuthology.orchestra.run.trial068.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-22T23:20:03.087 INFO:teuthology.orchestra.run.trial068.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-22T23:20:03.112 INFO:teuthology.orchestra.run.trial068.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-22T23:20:03.112 INFO:teuthology.orchestra.run.trial068.stdout:Get:79 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-22T23:20:03.172 INFO:teuthology.orchestra.run.trial068.stdout:Get:80 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-22T23:20:03.173 INFO:teuthology.orchestra.run.trial068.stdout:Get:81 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-22T23:20:03.173 INFO:teuthology.orchestra.run.trial068.stdout:Get:82 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-22T23:20:03.173 INFO:teuthology.orchestra.run.trial068.stdout:Get:83 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-22T23:20:03.177 INFO:teuthology.orchestra.run.trial068.stdout:Get:84 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-22T23:20:03.260 INFO:teuthology.orchestra.run.trial068.stdout:Get:85 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-22T23:20:03.261 INFO:teuthology.orchestra.run.trial068.stdout:Get:86 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-22T23:20:03.262 INFO:teuthology.orchestra.run.trial068.stdout:Get:87 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-22T23:20:03.266 INFO:teuthology.orchestra.run.trial068.stdout:Get:88 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-22T23:20:03.269 INFO:teuthology.orchestra.run.trial068.stdout:Get:89 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-22T23:20:03.348 INFO:teuthology.orchestra.run.trial068.stdout:Get:90 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-22T23:20:03.348 INFO:teuthology.orchestra.run.trial068.stdout:Get:91 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-22T23:20:03.354 INFO:teuthology.orchestra.run.trial068.stdout:Get:92 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-22T23:20:03.354 INFO:teuthology.orchestra.run.trial068.stdout:Get:93 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-22T23:20:03.358 INFO:teuthology.orchestra.run.trial068.stdout:Get:94 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-22T23:20:03.436 INFO:teuthology.orchestra.run.trial068.stdout:Get:95 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-22T23:20:03.437 INFO:teuthology.orchestra.run.trial068.stdout:Get:96 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-22T23:20:03.437 INFO:teuthology.orchestra.run.trial068.stdout:Get:97 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-22T23:20:03.438 INFO:teuthology.orchestra.run.trial068.stdout:Get:98 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-22T23:20:03.439 INFO:teuthology.orchestra.run.trial068.stdout:Get:99 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-22T23:20:03.524 INFO:teuthology.orchestra.run.trial068.stdout:Get:100 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-22T23:20:03.524 INFO:teuthology.orchestra.run.trial068.stdout:Get:101 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-22T23:20:03.524 INFO:teuthology.orchestra.run.trial068.stdout:Get:102 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-22T23:20:03.526 INFO:teuthology.orchestra.run.trial068.stdout:Get:103 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-22T23:20:03.527 INFO:teuthology.orchestra.run.trial068.stdout:Get:104 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-22T23:20:03.734 INFO:teuthology.orchestra.run.trial068.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-689-gf90f4eae-1jammy [135 kB] 2026-02-22T23:20:03.734 INFO:teuthology.orchestra.run.trial068.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-689-gf90f4eae-1jammy [41.0 kB] 2026-02-22T23:20:03.735 INFO:teuthology.orchestra.run.trial068.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-689-gf90f4eae-1jammy [13.7 MB] 2026-02-22T23:20:03.954 INFO:teuthology.orchestra.run.trial068.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-689-gf90f4eae-1jammy [92.3 kB] 2026-02-22T23:20:04.294 INFO:teuthology.orchestra.run.trial068.stdout:Fetched 178 MB in 3s (64.8 MB/s) 2026-02-22T23:20:04.310 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-22T23:20:04.471 INFO:teuthology.orchestra.run.trial068.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-22T23:20:04.473 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-22T23:20:04.475 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-22T23:20:04.496 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-22T23:20:04.508 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-22T23:20:04.508 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-22T23:20:04.526 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-22T23:20:04.537 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-22T23:20:04.538 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-22T23:20:04.557 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-22T23:20:04.565 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T23:20:04.570 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:04.602 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-22T23:20:04.610 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T23:20:04.611 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:04.627 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-22T23:20:04.636 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T23:20:04.636 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:04.659 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-22T23:20:04.667 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-22T23:20:04.667 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-22T23:20:04.683 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package librados2. 2026-02-22T23:20:04.691 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../007-librados2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:04.692 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking librados2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.732 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libnbd0. 2026-02-22T23:20:04.740 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-22T23:20:04.741 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-22T23:20:04.755 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package librbd1. 2026-02-22T23:20:04.763 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../009-librbd1_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:04.764 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking librbd1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.805 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libcephfs2. 2026-02-22T23:20:04.813 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:04.813 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libcephfs2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.834 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-rados. 2026-02-22T23:20:04.842 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../011-python3-rados_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:04.842 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-rados (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.859 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-22T23:20:04.868 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:04.868 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-ceph-argparse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.882 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-cephfs. 2026-02-22T23:20:04.890 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:04.891 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-cephfs (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.906 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-22T23:20:04.914 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:04.915 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:04.932 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-22T23:20:04.941 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-22T23:20:04.941 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-22T23:20:04.957 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-prettytable. 2026-02-22T23:20:04.965 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-22T23:20:04.965 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-22T23:20:04.979 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-rbd. 2026-02-22T23:20:04.987 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:04.988 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-rbd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:05.005 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-22T23:20:05.013 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-22T23:20:05.014 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-22T23:20:05.033 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-22T23:20:05.041 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-22T23:20:05.041 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-22T23:20:05.058 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-22T23:20:05.066 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-22T23:20:05.067 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-22T23:20:05.084 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package lua5.1. 2026-02-22T23:20:05.093 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-22T23:20:05.093 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-22T23:20:05.113 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package lua-any. 2026-02-22T23:20:05.121 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-22T23:20:05.122 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-22T23:20:05.135 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package zip. 2026-02-22T23:20:05.143 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-22T23:20:05.144 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking zip (3.0-12build2) ... 2026-02-22T23:20:05.161 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package unzip. 2026-02-22T23:20:05.169 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-22T23:20:05.169 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-22T23:20:05.187 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package luarocks. 2026-02-22T23:20:05.195 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-22T23:20:05.196 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-22T23:20:05.234 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package librgw2. 2026-02-22T23:20:05.242 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../026-librgw2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:05.242 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking librgw2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:05.313 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-rgw. 2026-02-22T23:20:05.321 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:05.322 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-rgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:05.337 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-22T23:20:05.345 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-22T23:20:05.345 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-22T23:20:05.360 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libradosstriper1. 2026-02-22T23:20:05.368 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:05.369 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libradosstriper1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:05.387 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-common. 2026-02-22T23:20:05.395 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../030-ceph-common_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:05.396 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:05.602 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-base. 2026-02-22T23:20:05.611 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../031-ceph-base_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:05.617 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-base (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:05.683 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-22T23:20:05.692 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-22T23:20:05.692 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-22T23:20:05.707 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-cheroot. 2026-02-22T23:20:05.716 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-22T23:20:05.716 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-22T23:20:05.733 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-22T23:20:05.741 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-22T23:20:05.742 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-22T23:20:05.756 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-22T23:20:05.764 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-22T23:20:05.765 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-22T23:20:05.779 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-22T23:20:05.787 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-22T23:20:05.787 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-22T23:20:05.802 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-tempora. 2026-02-22T23:20:05.810 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-22T23:20:05.811 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-22T23:20:05.826 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-portend. 2026-02-22T23:20:05.834 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-22T23:20:05.834 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-22T23:20:05.849 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-22T23:20:05.858 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-22T23:20:05.858 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-22T23:20:05.872 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-22T23:20:05.880 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-22T23:20:05.881 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-22T23:20:05.903 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-natsort. 2026-02-22T23:20:05.912 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-22T23:20:05.912 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-22T23:20:05.928 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-logutils. 2026-02-22T23:20:05.936 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-22T23:20:05.937 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-22T23:20:05.951 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-mako. 2026-02-22T23:20:05.960 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-22T23:20:05.960 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-22T23:20:05.977 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-22T23:20:05.986 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-22T23:20:05.986 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-22T23:20:06.001 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-22T23:20:06.009 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-22T23:20:06.009 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-22T23:20:06.023 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-webob. 2026-02-22T23:20:06.031 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-22T23:20:06.032 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-22T23:20:06.049 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-waitress. 2026-02-22T23:20:06.057 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-22T23:20:06.059 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-22T23:20:06.075 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-tempita. 2026-02-22T23:20:06.083 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-22T23:20:06.084 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-22T23:20:06.099 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-paste. 2026-02-22T23:20:06.108 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-22T23:20:06.108 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-22T23:20:06.136 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-22T23:20:06.144 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-22T23:20:06.144 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-22T23:20:06.159 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-22T23:20:06.167 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-22T23:20:06.167 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-22T23:20:06.183 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-webtest. 2026-02-22T23:20:06.191 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-22T23:20:06.191 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-22T23:20:06.207 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pecan. 2026-02-22T23:20:06.215 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-22T23:20:06.215 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-22T23:20:06.239 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-22T23:20:06.247 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-22T23:20:06.247 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-22T23:20:06.267 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-22T23:20:06.276 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:06.276 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mgr-modules-core (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.304 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-22T23:20:06.312 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.313 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.329 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mgr. 2026-02-22T23:20:06.337 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.337 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mgr (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.359 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mon. 2026-02-22T23:20:06.368 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.368 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mon (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.444 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-22T23:20:06.452 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-22T23:20:06.452 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-22T23:20:06.469 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-osd. 2026-02-22T23:20:06.477 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.477 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-osd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.633 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph. 2026-02-22T23:20:06.642 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../061-ceph_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.642 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.656 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-fuse. 2026-02-22T23:20:06.664 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.664 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.689 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mds. 2026-02-22T23:20:06.698 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.699 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mds (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.732 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package cephadm. 2026-02-22T23:20:06.740 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../064-cephadm_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:06.741 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.757 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-22T23:20:06.766 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-22T23:20:06.766 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-22T23:20:06.788 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-22T23:20:06.796 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:06.796 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mgr-cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:06.816 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-22T23:20:06.824 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-22T23:20:06.824 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-22T23:20:06.840 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-routes. 2026-02-22T23:20:06.848 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-22T23:20:06.848 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-22T23:20:06.868 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-22T23:20:06.876 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:06.876 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mgr-dashboard (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:07.080 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-22T23:20:07.089 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-22T23:20:07.089 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-22T23:20:07.129 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-joblib. 2026-02-22T23:20:07.137 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-22T23:20:07.137 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-22T23:20:07.164 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-22T23:20:07.172 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-22T23:20:07.173 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-22T23:20:07.188 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-sklearn. 2026-02-22T23:20:07.197 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-22T23:20:07.197 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-22T23:20:07.279 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-22T23:20:07.288 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:07.288 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:07.456 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-cachetools. 2026-02-22T23:20:07.465 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-22T23:20:07.465 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-22T23:20:07.480 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-rsa. 2026-02-22T23:20:07.488 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-22T23:20:07.489 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-22T23:20:07.506 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-google-auth. 2026-02-22T23:20:07.514 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-22T23:20:07.515 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-22T23:20:07.532 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-22T23:20:07.539 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-22T23:20:07.540 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-22T23:20:07.556 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-websocket. 2026-02-22T23:20:07.564 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-22T23:20:07.564 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-22T23:20:07.582 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-22T23:20:07.591 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-22T23:20:07.591 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-22T23:20:07.686 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-22T23:20:07.695 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:07.696 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:07.710 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-22T23:20:07.718 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-22T23:20:07.726 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-22T23:20:07.742 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-22T23:20:07.750 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-22T23:20:07.751 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-22T23:20:07.766 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package jq. 2026-02-22T23:20:07.775 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-22T23:20:07.775 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-22T23:20:07.790 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package socat. 2026-02-22T23:20:07.798 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-22T23:20:07.798 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-22T23:20:07.818 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package xmlstarlet. 2026-02-22T23:20:07.827 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-22T23:20:07.827 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-22T23:20:07.864 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-test. 2026-02-22T23:20:07.873 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../087-ceph-test_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:07.873 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-test (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:08.342 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package ceph-volume. 2026-02-22T23:20:08.350 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:08.350 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking ceph-volume (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:08.373 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-22T23:20:08.382 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:08.383 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking libcephfs-dev (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:08.399 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-22T23:20:08.408 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-22T23:20:08.408 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-22T23:20:08.430 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-22T23:20:08.439 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-22T23:20:08.439 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-22T23:20:08.456 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package nvme-cli. 2026-02-22T23:20:08.465 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-22T23:20:08.465 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-22T23:20:08.496 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package pkg-config. 2026-02-22T23:20:08.505 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-22T23:20:08.506 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-22T23:20:08.521 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-22T23:20:08.530 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-22T23:20:08.530 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-22T23:20:08.561 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-22T23:20:08.570 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-22T23:20:08.571 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-22T23:20:08.585 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pastescript. 2026-02-22T23:20:08.594 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-22T23:20:08.594 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-22T23:20:08.613 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pluggy. 2026-02-22T23:20:08.622 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-22T23:20:08.622 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-22T23:20:08.638 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-psutil. 2026-02-22T23:20:08.647 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-22T23:20:08.647 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-22T23:20:08.666 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-py. 2026-02-22T23:20:08.675 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-22T23:20:08.675 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-22T23:20:08.695 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pygments. 2026-02-22T23:20:08.704 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-22T23:20:08.704 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-22T23:20:08.746 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-22T23:20:08.755 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-22T23:20:08.755 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-22T23:20:08.771 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-toml. 2026-02-22T23:20:08.779 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-22T23:20:08.779 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-22T23:20:08.795 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-pytest. 2026-02-22T23:20:08.804 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-22T23:20:08.804 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-22T23:20:08.826 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package python3-simplejson. 2026-02-22T23:20:08.834 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-22T23:20:08.834 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-22T23:20:08.852 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-22T23:20:08.861 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-22T23:20:08.861 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-22T23:20:08.928 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package radosgw. 2026-02-22T23:20:08.937 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../106-radosgw_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:08.937 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking radosgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:09.049 INFO:teuthology.orchestra.run.trial068.stdout:Selecting previously unselected package rbd-fuse. 2026-02-22T23:20:09.058 INFO:teuthology.orchestra.run.trial068.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:09.058 INFO:teuthology.orchestra.run.trial068.stdout:Unpacking rbd-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:09.087 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-22T23:20:09.153 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-22T23:20:09.154 INFO:teuthology.orchestra.run.trial068.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-22T23:20:09.223 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:20:09.497 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-22T23:20:09.872 INFO:teuthology.orchestra.run.trial068.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-22T23:20:09.883 INFO:teuthology.orchestra.run.trial068.stdout:Setting up cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:09.926 INFO:teuthology.orchestra.run.trial068.stdout:Adding system user cephadm....done 2026-02-22T23:20:09.935 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-22T23:20:10.005 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-22T23:20:10.077 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-22T23:20:10.077 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-22T23:20:10.146 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-22T23:20:10.221 INFO:teuthology.orchestra.run.trial068.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-22T23:20:10.222 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-22T23:20:10.315 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-22T23:20:10.435 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-22T23:20:10.508 INFO:teuthology.orchestra.run.trial068.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-22T23:20:10.519 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-22T23:20:10.594 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-22T23:20:10.667 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-ceph-argparse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:10.734 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-22T23:20:10.735 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-22T23:20:10.736 INFO:teuthology.orchestra.run.trial068.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-22T23:20:10.737 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-22T23:20:10.737 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-22T23:20:10.738 INFO:teuthology.orchestra.run.trial068.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-22T23:20:10.755 INFO:teuthology.orchestra.run.trial068.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-22T23:20:10.757 INFO:teuthology.orchestra.run.trial068.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-22T23:20:10.758 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-22T23:20:10.759 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-22T23:20:10.882 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-22T23:20:10.958 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-22T23:20:11.035 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-22T23:20:11.121 INFO:teuthology.orchestra.run.trial068.stdout:Setting up zip (3.0-12build2) ... 2026-02-22T23:20:11.121 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-22T23:20:11.375 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-22T23:20:11.449 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-22T23:20:11.450 INFO:teuthology.orchestra.run.trial068.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-22T23:20:11.451 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-22T23:20:11.543 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-22T23:20:11.665 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-22T23:20:11.788 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-22T23:20:11.871 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-22T23:20:11.976 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-22T23:20:12.046 INFO:teuthology.orchestra.run.trial068.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-22T23:20:12.046 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:12.130 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-22T23:20:12.630 INFO:teuthology.orchestra.run.trial068.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-22T23:20:12.652 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:12.657 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-22T23:20:12.730 INFO:teuthology.orchestra.run.trial068.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-22T23:20:12.731 INFO:teuthology.orchestra.run.trial068.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-22T23:20:12.732 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-22T23:20:12.805 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-22T23:20:12.876 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:12.877 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-22T23:20:12.951 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-22T23:20:13.024 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-22T23:20:13.099 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-22T23:20:13.171 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-22T23:20:13.241 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-22T23:20:13.319 INFO:teuthology.orchestra.run.trial068.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-22T23:20:13.320 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-22T23:20:13.404 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-22T23:20:13.405 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-22T23:20:13.483 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-22T23:20:13.564 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-22T23:20:13.646 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-22T23:20:13.720 INFO:teuthology.orchestra.run.trial068.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-22T23:20:13.721 INFO:teuthology.orchestra.run.trial068.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-22T23:20:13.722 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-22T23:20:13.723 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-22T23:20:13.857 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-22T23:20:13.935 INFO:teuthology.orchestra.run.trial068.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-22T23:20:13.936 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-22T23:20:14.001 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:14.002 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-22T23:20:14.088 INFO:teuthology.orchestra.run.trial068.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-22T23:20:14.090 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-22T23:20:14.169 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-22T23:20:14.297 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-22T23:20:14.384 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-22T23:20:14.495 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-22T23:20:14.495 INFO:teuthology.orchestra.run.trial068.stdout:Setting up librados2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:14.496 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libsqlite3-mod-ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:14.497 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-22T23:20:15.017 INFO:teuthology.orchestra.run.trial068.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-22T23:20:15.030 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libcephfs2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.031 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libradosstriper1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.032 INFO:teuthology.orchestra.run.trial068.stdout:Setting up librbd1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.032 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mgr-modules-core (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.033 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.102 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-22T23:20:15.102 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-22T23:20:15.496 INFO:teuthology.orchestra.run.trial068.stdout:Setting up libcephfs-dev (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.497 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-rados (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.498 INFO:teuthology.orchestra.run.trial068.stdout:Setting up librgw2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.499 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-rbd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.500 INFO:teuthology.orchestra.run.trial068.stdout:Setting up rbd-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.500 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-rgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.501 INFO:teuthology.orchestra.run.trial068.stdout:Setting up python3-cephfs (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.502 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:15.531 INFO:teuthology.orchestra.run.trial068.stdout:Adding group ceph....done 2026-02-22T23:20:15.567 INFO:teuthology.orchestra.run.trial068.stdout:Adding system user ceph....done 2026-02-22T23:20:15.578 INFO:teuthology.orchestra.run.trial068.stdout:Setting system user ceph properties....done 2026-02-22T23:20:15.584 INFO:teuthology.orchestra.run.trial068.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-22T23:20:15.670 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-22T23:20:15.943 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-22T23:20:16.295 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-test (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:16.296 INFO:teuthology.orchestra.run.trial068.stdout:Setting up radosgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:16.531 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:20:16.532 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:20:16.881 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-base (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:16.998 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-22T23:20:17.343 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mds (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:17.402 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-22T23:20:17.402 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-22T23:20:17.771 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mgr (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:17.850 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-22T23:20:17.850 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-22T23:20:18.218 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-osd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:18.310 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-22T23:20:18.310 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-22T23:20:18.684 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mgr-k8sevents (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:18.685 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:18.695 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mon (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:18.767 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-22T23:20:18.767 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-22T23:20:19.139 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mgr-cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:19.150 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:19.151 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-mgr-dashboard (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:19.163 INFO:teuthology.orchestra.run.trial068.stdout:Setting up ceph-volume (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:19.272 INFO:teuthology.orchestra.run.trial068.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-22T23:20:19.279 INFO:teuthology.orchestra.run.trial068.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-22T23:20:19.526 INFO:teuthology.orchestra.run.trial068.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-22T23:20:19.610 INFO:teuthology.orchestra.run.trial068.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-22T23:20:20.110 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:20.110 INFO:teuthology.orchestra.run.trial068.stdout:Running kernel seems to be up-to-date. 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout:The processor microcode seems to be up-to-date. 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout:No services need to be restarted. 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout:No containers need to be restarted. 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout:No user sessions are running outdated binaries. 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:20.111 INFO:teuthology.orchestra.run.trial068.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-22T23:20:20.915 INFO:teuthology.orchestra.run.trial068.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-22T23:20:20.920 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:20.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:20:21.091 DEBUG:teuthology.orchestra.run.trial068:> dpkg-query -W -f '${Version}' ceph 2026-02-22T23:20:21.101 INFO:teuthology.orchestra.run.trial068.stdout:19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:20:21.102 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:20:21.102 INFO:teuthology.task.install:The correct ceph version 19.2.3-689-gf90f4eae-1jammy is installed. 2026-02-22T23:20:21.104 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-22T23:20:21.104 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:21.104 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:20:21.151 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-22T23:20:21.151 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:21.151 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:20:21.203 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:20:21.254 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-22T23:20:21.254 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:21.254 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:20:21.303 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:20:21.354 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-22T23:20:21.354 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:21.355 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:20:21.405 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:20:21.453 INFO:teuthology.run_tasks:Running task ceph... 2026-02-22T23:20:21.543 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-22T23:20:21.543 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod 777 /var/log/ceph 2026-02-22T23:20:21.551 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-22T23:20:21.552 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T23:20:21.602 INFO:tasks.ceph:Creating extra log directories... 2026-02-22T23:20:21.602 DEBUG:teuthology.orchestra.run.trial068:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T23:20:21.656 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-22T23:20:21.656 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-22T23:20:21.657 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379', 'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'description': 'rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '64379', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 5907, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'targets': {'trial068.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK7DJMO3UkcSK6rdHn+1TRWzJUhsIgwF2qrB516Dx0ZYDUgJUfKVHrKRzPUXq8OtDZc7XTnGzZDrjfxEKJ9N8nU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'cluster': 'ceph'}}, {'divergent_priors': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-22_19:35:40', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653'} 2026-02-22T23:20:21.657 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T23:20:21.693 DEBUG:teuthology.orchestra.run.trial068:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T23:20:21.742 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:21.742 DEBUG:teuthology.orchestra.run.trial068:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T23:20:21.786 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T23:20:21.786 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_1 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout:Device: 5h/5d Inode: 916 Links: 1 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-22 23:19:39.423502499 +0000 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-22 23:19:39.267505153 +0000 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-22 23:19:39.267505153 +0000 2026-02-22T23:20:21.830 INFO:teuthology.orchestra.run.trial068.stdout: Birth: - 2026-02-22T23:20:21.830 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T23:20:21.877 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-22T23:20:21.878 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-22T23:20:21.878 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000200814 s, 2.5 MB/s 2026-02-22T23:20:21.879 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T23:20:21.928 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_2 2026-02-22T23:20:21.974 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout:Device: 5h/5d Inode: 928 Links: 1 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-22 23:19:39.775496516 +0000 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-22 23:19:39.587499711 +0000 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-22 23:19:39.587499711 +0000 2026-02-22T23:20:21.975 INFO:teuthology.orchestra.run.trial068.stdout: Birth: - 2026-02-22T23:20:21.976 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T23:20:22.025 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-22T23:20:22.025 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-22T23:20:22.025 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000174856 s, 2.9 MB/s 2026-02-22T23:20:22.026 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T23:20:22.071 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_3 2026-02-22T23:20:22.118 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T23:20:22.118 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:20:22.118 INFO:teuthology.orchestra.run.trial068.stdout:Device: 5h/5d Inode: 963 Links: 1 2026-02-22T23:20:22.119 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:20:22.119 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-22 23:19:39.931493866 +0000 2026-02-22T23:20:22.119 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-22 23:19:39.927493934 +0000 2026-02-22T23:20:22.119 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-22 23:19:39.927493934 +0000 2026-02-22T23:20:22.119 INFO:teuthology.orchestra.run.trial068.stdout: Birth: - 2026-02-22T23:20:22.119 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T23:20:22.165 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-22T23:20:22.165 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-22T23:20:22.165 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000185355 s, 2.8 MB/s 2026-02-22T23:20:22.166 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T23:20:22.212 DEBUG:teuthology.orchestra.run.trial068:> stat /dev/vg_nvme/lv_4 2026-02-22T23:20:22.257 INFO:teuthology.orchestra.run.trial068.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout:Device: 5h/5d Inode: 975 Links: 1 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout:Access: 2026-02-22 23:19:40.455484959 +0000 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout:Modify: 2026-02-22 23:19:40.299487609 +0000 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout:Change: 2026-02-22 23:19:40.299487609 +0000 2026-02-22T23:20:22.258 INFO:teuthology.orchestra.run.trial068.stdout: Birth: - 2026-02-22T23:20:22.258 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T23:20:22.305 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records in 2026-02-22T23:20:22.305 INFO:teuthology.orchestra.run.trial068.stderr:1+0 records out 2026-02-22T23:20:22.305 INFO:teuthology.orchestra.run.trial068.stderr:512 bytes copied, 0.000164055 s, 3.1 MB/s 2026-02-22T23:20:22.306 DEBUG:teuthology.orchestra.run.trial068:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T23:20:22.351 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:20:22.351 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial068.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-22T23:20:22.351 INFO:tasks.ceph:Generating config... 2026-02-22T23:20:22.354 INFO:tasks.ceph:[global] mon election default strategy = 3 2026-02-22T23:20:22.354 INFO:tasks.ceph:[global] ms bind msgr1 = False 2026-02-22T23:20:22.354 INFO:tasks.ceph:[global] ms bind msgr2 = True 2026-02-22T23:20:22.355 INFO:tasks.ceph:[global] ms type = async 2026-02-22T23:20:22.355 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-22T23:20:22.355 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-22T23:20:22.355 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-22T23:20:22.355 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-22T23:20:22.355 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-22T23:20:22.355 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-22T23:20:22.355 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-22T23:20:22.355 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2026-02-22T23:20:22.355 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2026-02-22T23:20:22.355 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] debug bluefs = 20 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] debug bluestore = 20 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-22T23:20:22.356 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-22T23:20:22.357 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-22T23:20:22.357 INFO:tasks.ceph:Setting up mon.a... 2026-02-22T23:20:22.358 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-22T23:20:22.411 INFO:teuthology.orchestra.run.trial068.stdout:creating /etc/ceph/ceph.keyring 2026-02-22T23:20:22.415 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-22T23:20:22.478 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:20:22.527 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.68')] 2026-02-22T23:20:22.528 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.68', 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-22T23:20:22.529 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:22.529 DEBUG:teuthology.orchestra.run.trial068:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T23:20:22.575 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.68 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:monmaptool: generated fsid c2872058-bff0-4756-bb48-18a697ed2e43 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:setting min_mon_release = quincy 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:epoch 0 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:fsid c2872058-bff0-4756-bb48-18a697ed2e43 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:last_changed 2026-02-22T23:20:22.634950+0000 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:created 2026-02-22T23:20:22.634950+0000 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:min_mon_release 17 (quincy) 2026-02-22T23:20:22.635 INFO:teuthology.orchestra.run.trial068.stdout:election_strategy: 3 2026-02-22T23:20:22.636 INFO:teuthology.orchestra.run.trial068.stdout:0: [v2:10.20.193.68:3300/0,v1:10.20.193.68:6789/0] mon.a 2026-02-22T23:20:22.636 INFO:teuthology.orchestra.run.trial068.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-22T23:20:22.637 DEBUG:teuthology.orchestra.run.trial068:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T23:20:22.682 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c2872058-bff0-4756-bb48-18a697ed2e43... 2026-02-22T23:20:22.683 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout:[global] 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: chdir = "" 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: auth supported = cephx 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: filestore xattr use omap = true 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.742 INFO:teuthology.orchestra.run.trial068.stdout: mon clock drift allowed = 1.000 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: osd crush chooseleaf type = 0 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: auth debug = true 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: ms die on old message = true 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: ms die on bug = true 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: mon pg warn max object skew = 0 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T23:20:22.743 INFO:teuthology.orchestra.run.trial068.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: osd pool default size = 2 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon osd allow primary affinity = true 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon osd allow pg remap = true 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on legacy crush tunables = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on crush straw calc version zero = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on no sortbitwise = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on osd down out interval zero = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon warn on too few osds = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T23:20:22.744 INFO:teuthology.orchestra.run.trial068.stdout: mon_allow_pool_size_one = true 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: osd default data pool replay window = 5 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: mon allow pool delete = true 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: mon cluster log file level = debug 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: debug asserts on shutdown = true 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: mon health detail to clog = false 2026-02-22T23:20:22.745 INFO:teuthology.orchestra.run.trial068.stdout: mon host = 10.20.193.68 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: mon election default strategy = 3 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: ms bind msgr1 = False 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: ms bind msgr2 = True 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: ms type = async 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: fsid = c2872058-bff0-4756-bb48-18a697ed2e43 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout:[osd] 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: osd journal size = 100 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: osd scrub load threshold = 5.0 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: osd scrub max interval = 600 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: osd mclock profile = high_recovery_ops 2026-02-22T23:20:22.746 INFO:teuthology.orchestra.run.trial068.stdout: osd mclock skip benchmark = True 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd recover clone overlap = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd recovery max chunk = 1048576 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd debug shutdown = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd debug op order = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd debug verify stray on activate = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd debug trim objects = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd open classes on start = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd debug pg log writeout = true 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.747 INFO:teuthology.orchestra.run.trial068.stdout: osd deep scrub update digest min age = 30 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: osd map max advance = 10 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: journal zero on create = true 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: filestore ondisk finisher threads = 3 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: filestore apply finisher threads = 3 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: bdev debug aio = true 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: osd debug misdirected ops = true 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: bluestore block size = 96636764160 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: bluestore compression algorithm = lz4 2026-02-22T23:20:22.748 INFO:teuthology.orchestra.run.trial068.stdout: bluestore compression mode = aggressive 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: bluestore fsck on mount = True 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: bluestore zero block detection = True 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: debug bluefs = 20 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: debug bluestore = 20 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: debug ms = 1 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: debug osd = 20 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: debug rocksdb = 10 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: mon osd full ratio = 0.9 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: mon osd nearfull ratio = 0.8 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: osd debug verify cached snaps = True 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: osd debug verify missing on start = True 2026-02-22T23:20:22.749 INFO:teuthology.orchestra.run.trial068.stdout: osd failsafe full ratio = 0.95 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: osd mclock override recovery settings = True 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: osd objectstore = bluestore 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: osd op queue = debug_random 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: osd op queue cut off = debug_random 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout:[mgr] 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: debug ms = 1 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: debug mgr = 20 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: debug mon = 20 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: debug auth = 20 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:20:22.750 INFO:teuthology.orchestra.run.trial068.stdout: mgr/telemetry/nag = false 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout:[mon] 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: debug ms = 1 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: debug mon = 20 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: debug paxos = 20 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: debug auth = 20 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: mon data avail warn = 5 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: mon mgr mkfs grace = 240 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: mon osd reporter subtree level = osd 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: mon osd prime pg temp = true 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:20:22.751 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: # 1m isn't quite enough 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: mon_down_mkfs_grace = 2m 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: mon_warn_on_filestore_osds = false 2026-02-22T23:20:22.752 INFO:teuthology.orchestra.run.trial068.stdout: mon scrub interval = 300 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout:[client] 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout: rgw cache enabled = true 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout: rgw enable ops log = true 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout: rgw enable usage log = true 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T23:20:22.753 INFO:teuthology.orchestra.run.trial068.stdout:[mon.a] 2026-02-22T23:20:22.754 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-22T23:20:22.755 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-22T23:20:22.817 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-22T23:20:22.817 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:22.817 DEBUG:teuthology.orchestra.run.trial068:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-22T23:20:22.861 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:22.861 DEBUG:teuthology.orchestra.run.trial068:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-22T23:20:22.905 INFO:tasks.ceph:Sending monmap to node ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:20:22.906 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:22.906 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T23:20:22.906 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:20:22.960 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:22.961 DEBUG:teuthology.orchestra.run.trial068:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:23.006 INFO:tasks.ceph:Setting up mon nodes... 2026-02-22T23:20:23.006 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-22T23:20:23.006 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T23:20:23.072 INFO:teuthology.orchestra.run.trial068.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T23:20:23.074 INFO:tasks.ceph:Setting up mds nodes... 2026-02-22T23:20:23.074 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-22T23:20:23.074 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-22T23:20:23.131 INFO:teuthology.orchestra.run.trial068.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-22T23:20:23.138 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-22T23:20:23.138 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial068.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-22T23:20:23.138 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-22T23:20:23.186 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:20:23.186 INFO:tasks.ceph:role: osd.0 2026-02-22T23:20:23.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:20:23.186 DEBUG:teuthology.orchestra.run.trial068:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:23.236 INFO:teuthology.orchestra.run.trial068.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:23.826 INFO:teuthology.orchestra.run.trial068.stdout:Discarding blocks...Done. 2026-02-22T23:20:23.828 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial068.front.sepia.ceph.com -o noatime 2026-02-22T23:20:23.828 DEBUG:teuthology.orchestra.run.trial068:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-22T23:20:23.887 DEBUG:teuthology.orchestra.run.trial068:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-22T23:20:23.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:20:23.937 INFO:teuthology.orchestra.run.trial068.stderr:sudo: /sbin/restorecon: command not found 2026-02-22T23:20:23.937 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:23.987 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:20:23.987 INFO:tasks.ceph:role: osd.1 2026-02-22T23:20:23.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:20:23.987 DEBUG:teuthology.orchestra.run.trial068:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:24.041 INFO:teuthology.orchestra.run.trial068.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:24.042 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:24.042 INFO:teuthology.orchestra.run.trial068.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:24.610 INFO:teuthology.orchestra.run.trial068.stdout:Discarding blocks...Done. 2026-02-22T23:20:24.611 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial068.front.sepia.ceph.com -o noatime 2026-02-22T23:20:24.611 DEBUG:teuthology.orchestra.run.trial068:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:24.668 DEBUG:teuthology.orchestra.run.trial068:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:24.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:20:24.717 INFO:teuthology.orchestra.run.trial068.stderr:sudo: /sbin/restorecon: command not found 2026-02-22T23:20:24.717 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:24.767 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:20:24.767 INFO:tasks.ceph:role: osd.2 2026-02-22T23:20:24.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:20:24.767 DEBUG:teuthology.orchestra.run.trial068:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T23:20:24.820 INFO:teuthology.orchestra.run.trial068.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:24.820 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:24.820 INFO:teuthology.orchestra.run.trial068.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:24.820 INFO:teuthology.orchestra.run.trial068.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-22T23:20:24.820 INFO:teuthology.orchestra.run.trial068.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:24.821 INFO:teuthology.orchestra.run.trial068.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:24.821 INFO:teuthology.orchestra.run.trial068.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:24.821 INFO:teuthology.orchestra.run.trial068.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:24.821 INFO:teuthology.orchestra.run.trial068.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:24.821 INFO:teuthology.orchestra.run.trial068.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:25.391 INFO:teuthology.orchestra.run.trial068.stdout:Discarding blocks...Done. 2026-02-22T23:20:25.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial068.front.sepia.ceph.com -o noatime 2026-02-22T23:20:25.393 DEBUG:teuthology.orchestra.run.trial068:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:25.452 DEBUG:teuthology.orchestra.run.trial068:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:25.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:20:25.501 INFO:teuthology.orchestra.run.trial068.stderr:sudo: /sbin/restorecon: command not found 2026-02-22T23:20:25.501 DEBUG:teuthology.orchestra.run.trial068:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:25.564 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:25.562+0000 7f0a047058c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-22T23:20:25.564 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:25.562+0000 7f0a047058c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-22T23:20:25.564 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:25.562+0000 7f0a047058c0 -1 bdev(0x55f44d70d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:25.564 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:25.562+0000 7f0a047058c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-22T23:20:28.662 DEBUG:teuthology.orchestra.run.trial068:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-22T23:20:28.711 DEBUG:teuthology.orchestra.run.trial068:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:28.775 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:28.774+0000 7f50da9088c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-22T23:20:28.776 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:28.774+0000 7f50da9088c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-22T23:20:28.776 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:28.774+0000 7f50da9088c0 -1 bdev(0x55c977047800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:28.776 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:28.774+0000 7f50da9088c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-22T23:20:31.862 DEBUG:teuthology.orchestra.run.trial068:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:31.911 DEBUG:teuthology.orchestra.run.trial068:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:31.975 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:31.974+0000 7f0c00dbd8c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-22T23:20:31.975 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:31.974+0000 7f0c00dbd8c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-22T23:20:31.975 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:31.974+0000 7f0c00dbd8c0 -1 bdev(0x55c68983f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:31.975 INFO:teuthology.orchestra.run.trial068.stderr:2026-02-22T23:20:31.974+0000 7f0c00dbd8c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-22T23:20:35.071 DEBUG:teuthology.orchestra.run.trial068:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:35.123 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-22T23:20:35.123 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:35.123 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-22T23:20:35.175 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:35.176 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-22T23:20:35.228 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:35.228 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-22T23:20:35.279 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:35.279 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-22T23:20:35.330 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:35.330 DEBUG:teuthology.orchestra.run.trial068:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-22T23:20:35.379 INFO:tasks.ceph:Adding keys to all mons... 2026-02-22T23:20:35.380 DEBUG:teuthology.orchestra.run.trial068:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T23:20:35.428 INFO:teuthology.orchestra.run.trial068.stdout:[mgr.x] 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout: key = AQA3j5tpCGhOBBAAN3k75tzgjqwhfFqYTlqjaw== 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout:[osd.0] 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout: key = AQA5j5tpl8elIRAAPAhcNwcSUPBsXiiO/3ykbw== 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout:[osd.1] 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout: key = AQA8j5tp7rg9LhAAOCAg6m/rqlTyxxYt9MZh+A== 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout:[osd.2] 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout: key = AQA/j5tpUKQjOhAALq1amiO6EvxACxAhWDUhNw== 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout:[client.0] 2026-02-22T23:20:35.429 INFO:teuthology.orchestra.run.trial068.stdout: key = AQA3j5tpNYrVBxAASodFdLJK0UzlE7P12sl8pw== 2026-02-22T23:20:35.431 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:20:35.496 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:20:35.560 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:20:35.623 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:20:35.687 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-22T23:20:35.752 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-22T23:20:35.752 DEBUG:teuthology.orchestra.run.trial068:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-22T23:20:35.802 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T23:20:35.867 DEBUG:teuthology.orchestra.run.trial068:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-22T23:20:35.918 DEBUG:teuthology.orchestra.run.trial068:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:35.961 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-22T23:20:35.962 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-22T23:20:35.962 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-22T23:20:36.003 INFO:tasks.ceph.mon.a:Started 2026-02-22T23:20:36.004 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph config set mgr mgr_pool false --force 2026-02-22T23:20:36.139 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-22T23:20:36.140 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-22T23:20:36.140 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-22T23:20:36.142 INFO:tasks.ceph.mgr.x:Started 2026-02-22T23:20:36.142 INFO:tasks.ceph:Setting crush tunables to default 2026-02-22T23:20:36.142 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd crush tunables default 2026-02-22T23:20:36.246 INFO:teuthology.orchestra.run.trial068.stderr:adjusted tunables profile to default 2026-02-22T23:20:36.248 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.246+0000 7fb650231280 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-22T23:20:36.256 INFO:tasks.ceph:check_enable_crimson: False 2026-02-22T23:20:36.256 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-22T23:20:36.256 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:36.256 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-22T23:20:36.264 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:36.265 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-22T23:20:36.274 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.270+0000 7fb650231280 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-22T23:20:36.282 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-22T23:20:36.282 DEBUG:teuthology.orchestra.run.trial068:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-22T23:20:36.312 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.310+0000 7fb650231280 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-22T23:20:36.318 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new 6498852d-91da-4351-9faa-b98ee637096f 0 2026-02-22T23:20:36.382 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.378+0000 7fb650231280 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-22T23:20:36.458 INFO:teuthology.orchestra.run.trial068.stdout:0 2026-02-22T23:20:36.462 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.458+0000 7fb650231280 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:20:36.467 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new 36291856-799f-464f-9583-431e83696b09 1 2026-02-22T23:20:36.500 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.498+0000 7fb650231280 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-22T23:20:36.576 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.574+0000 7fb650231280 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-22T23:20:36.599 INFO:teuthology.orchestra.run.trial068.stdout:1 2026-02-22T23:20:36.608 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd new c5f32d0e-9755-48db-a210-680579d30eb7 2 2026-02-22T23:20:36.706 INFO:teuthology.orchestra.run.trial068.stdout:2 2026-02-22T23:20:36.715 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-22T23:20:36.715 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-22T23:20:36.716 INFO:tasks.ceph.osd.0:Started 2026-02-22T23:20:36.716 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-22T23:20:36.716 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-22T23:20:36.717 INFO:tasks.ceph.osd.1:Started 2026-02-22T23:20:36.717 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T23:20:36.717 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-22T23:20:36.718 INFO:tasks.ceph.osd.2:Started 2026-02-22T23:20:36.718 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:20:36.750 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:36.746+0000 7fb650231280 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-22T23:20:36.809 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:36.809 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":5,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:20:36.705744+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:20:36.819 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-22T23:20:36.819 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-22T23:20:37.056 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.054+0000 7fb650231280 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-22T23:20:37.121 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:20:37.151 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.150+0000 7fb650231280 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-22T23:20:37.179 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.178+0000 7fb650231280 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-22T23:20:37.207 INFO:teuthology.misc.health.trial068.stdout: 2026-02-22T23:20:37.207 INFO:teuthology.misc.health.trial068.stdout:{"epoch":5,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:20:36.705744+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:20:37.211 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.210+0000 7fb650231280 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-22T23:20:37.215 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-22T23:20:37.256 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.254+0000 7fb650231280 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-22T23:20:37.361 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.358+0000 7fb650231280 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:20:37.559 INFO:tasks.ceph.mgr.x.trial068.stderr:ERROR:root:Module 'xmltodict' is not installed. 2026-02-22T23:20:37.738 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.734+0000 7fb650231280 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-22T23:20:37.771 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.770+0000 7fb650231280 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-22T23:20:37.867 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.866+0000 7fb650231280 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-22T23:20:37.893 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.890+0000 7fb650231280 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-22T23:20:37.920 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.918+0000 7fb650231280 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-22T23:20:37.948 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:37.946+0000 7fb650231280 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-22T23:20:38.070 INFO:tasks.ceph.mgr.x.trial068.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-22T23:20:38.070 INFO:tasks.ceph.mgr.x.trial068.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-22T23:20:38.070 INFO:tasks.ceph.mgr.x.trial068.stderr: from numpy import show_config as show_numpy_config 2026-02-22T23:20:38.077 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:38.074+0000 7fb650231280 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-22T23:20:38.104 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:38.102+0000 7fb650231280 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-22T23:20:38.144 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:38.142+0000 7fb650231280 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-22T23:20:38.280 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:20:38.278+0000 7f1d12c0f8c0 -1 Falling back to public interface 2026-02-22T23:20:38.512 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:20:38.510+0000 7f3618c958c0 -1 Falling back to public interface 2026-02-22T23:20:38.740 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:20:38.738+0000 7fa1f6eb48c0 -1 Falling back to public interface 2026-02-22T23:20:39.171 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:20:39.166+0000 7fb56e981640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-22T23:20:42.143 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:20:42.138+0000 7f1d12c0f8c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:20:42.804 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:20:42.802+0000 7fa1f6eb48c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:20:42.805 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:20:42.802+0000 7f3618c958c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:20:43.179 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:20:43.174+0000 7f1d0ebb8640 -1 osd.2 0 waiting for initial osdmap 2026-02-22T23:20:43.181 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:20:43.178+0000 7f1d0a9f3640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:20:43.517 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:20:43.662 INFO:teuthology.misc.health.trial068.stdout: 2026-02-22T23:20:43.663 INFO:teuthology.misc.health.trial068.stdout:{"epoch":7,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:20:43.171227+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:20:43.672 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-22T23:20:44.178 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:20:44.174+0000 7fa1f2e5d640 -1 osd.0 0 waiting for initial osdmap 2026-02-22T23:20:44.178 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:20:44.174+0000 7f3614c3e640 -1 osd.1 0 waiting for initial osdmap 2026-02-22T23:20:44.181 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:20:44.178+0000 7f3610a79640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:20:44.181 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:20:44.178+0000 7fa1eec98640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:20:49.974 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:20:50.112 INFO:teuthology.misc.health.trial068.stdout: 2026-02-22T23:20:50.113 INFO:teuthology.misc.health.trial068.stdout:{"epoch":9,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:20:45.176408+0000","last_up_change":"2026-02-22T23:20:45.176408+0000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":4036091367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":4036091367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6812","nonce":4036091367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6811","nonce":4036091367}]},"public_addr":"10.20.193.68:6809/4036091367","cluster_addr":"10.20.193.68:6810/4036091367","heartbeat_back_addr":"10.20.193.68:6812/4036091367","heartbeat_front_addr":"10.20.193.68:6811/4036091367","state":["exists","up"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":968320453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6806","nonce":968320453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":968320453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":968320453}]},"public_addr":"10.20.193.68:6805/968320453","cluster_addr":"10.20.193.68:6806/968320453","heartbeat_back_addr":"10.20.193.68:6808/968320453","heartbeat_front_addr":"10.20.193.68:6807/968320453","state":["exists","up"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6801","nonce":4010237467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":4010237467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":4010237467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":4010237467}]},"public_addr":"10.20.193.68:6801/4010237467","cluster_addr":"10.20.193.68:6802/4010237467","heartbeat_back_addr":"10.20.193.68:6804/4010237467","heartbeat_front_addr":"10.20.193.68:6803/4010237467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:20:50.121 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-22T23:20:50.121 INFO:tasks.ceph:Creating RBD pool 2026-02-22T23:20:50.121 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-22T23:20:51.165 INFO:teuthology.orchestra.run.trial068.stderr:pool 'rbd' created 2026-02-22T23:20:51.179 DEBUG:teuthology.orchestra.run.trial068:> rbd --cluster ceph pool init rbd 2026-02-22T23:20:54.181 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-22T23:20:54.182 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-22T23:20:54.182 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-22T23:20:54.182 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-22T23:20:54.183 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-22T23:20:54.345 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:54.354 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6800","nonce":1694231751}]},"active_addr":"10.20.193.68:6800/1694231751","active_change":"2026-02-22T23:20:38.163944+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":350649907}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":2166779734}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.68:0","nonce":3099693865}]}]} 2026-02-22T23:20:54.361 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-22T23:20:54.361 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-22T23:20:54.361 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:20:54.502 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:54.502 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":13,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:20:54.170381+0000","last_up_change":"2026-02-22T23:20:45.176408+0000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T23:20:50.267994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":4036091367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":4036091367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6812","nonce":4036091367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6811","nonce":4036091367}]},"public_addr":"10.20.193.68:6809/4036091367","cluster_addr":"10.20.193.68:6810/4036091367","heartbeat_back_addr":"10.20.193.68:6812/4036091367","heartbeat_front_addr":"10.20.193.68:6811/4036091367","state":["exists","up"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":968320453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6806","nonce":968320453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":968320453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":968320453}]},"public_addr":"10.20.193.68:6805/968320453","cluster_addr":"10.20.193.68:6806/968320453","heartbeat_back_addr":"10.20.193.68:6808/968320453","heartbeat_front_addr":"10.20.193.68:6807/968320453","state":["exists","up"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6801","nonce":4010237467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":4010237467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":4010237467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":4010237467}]},"public_addr":"10.20.193.68:6801/4010237467","cluster_addr":"10.20.193.68:6802/4010237467","heartbeat_back_addr":"10.20.193.68:6804/4010237467","heartbeat_front_addr":"10.20.193.68:6803/4010237467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:20:54.512 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-22T23:20:54.512 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:20:54.653 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:54.653 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":13,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:20:54.170381+0000","last_up_change":"2026-02-22T23:20:45.176408+0000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T23:20:50.267994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":4036091367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":4036091367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6812","nonce":4036091367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6811","nonce":4036091367}]},"public_addr":"10.20.193.68:6809/4036091367","cluster_addr":"10.20.193.68:6810/4036091367","heartbeat_back_addr":"10.20.193.68:6812/4036091367","heartbeat_front_addr":"10.20.193.68:6811/4036091367","state":["exists","up"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":968320453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6806","nonce":968320453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":968320453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":968320453}]},"public_addr":"10.20.193.68:6805/968320453","cluster_addr":"10.20.193.68:6806/968320453","heartbeat_back_addr":"10.20.193.68:6808/968320453","heartbeat_front_addr":"10.20.193.68:6807/968320453","state":["exists","up"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6801","nonce":4010237467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":4010237467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":4010237467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":4010237467}]},"public_addr":"10.20.193.68:6801/4010237467","cluster_addr":"10.20.193.68:6802/4010237467","heartbeat_back_addr":"10.20.193.68:6804/4010237467","heartbeat_front_addr":"10.20.193.68:6803/4010237467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:20:54.664 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T23:20:54.664 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T23:20:54.665 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T23:20:54.748 INFO:teuthology.orchestra.run.trial068.stdout:34359738371 2026-02-22T23:20:54.748 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:20:54.750 INFO:teuthology.orchestra.run.trial068.stdout:38654705668 2026-02-22T23:20:54.750 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:20:54.750 INFO:teuthology.orchestra.run.trial068.stdout:38654705667 2026-02-22T23:20:54.751 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:20:54.897 INFO:teuthology.orchestra.run.trial068.stdout:34359738370 2026-02-22T23:20:54.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.2 2026-02-22T23:20:54.937 INFO:teuthology.orchestra.run.trial068.stdout:38654705666 2026-02-22T23:20:54.947 INFO:teuthology.orchestra.run.trial068.stdout:38654705667 2026-02-22T23:20:54.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2026-02-22T23:20:54.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2026-02-22T23:20:55.911 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:20:55.950 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:20:55.959 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:20:56.070 INFO:teuthology.orchestra.run.trial068.stdout:34359738370 2026-02-22T23:20:56.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.2 2026-02-22T23:20:56.098 INFO:teuthology.orchestra.run.trial068.stdout:38654705666 2026-02-22T23:20:56.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2026-02-22T23:20:56.143 INFO:teuthology.orchestra.run.trial068.stdout:38654705667 2026-02-22T23:20:56.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2026-02-22T23:20:57.082 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:20:57.112 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:20:57.156 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:20:57.231 INFO:teuthology.orchestra.run.trial068.stdout:34359738372 2026-02-22T23:20:57.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738372 for osd.2 2026-02-22T23:20:57.243 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:57.273 INFO:teuthology.orchestra.run.trial068.stdout:38654705667 2026-02-22T23:20:57.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.1 2026-02-22T23:20:57.286 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:57.318 INFO:teuthology.orchestra.run.trial068.stdout:38654705668 2026-02-22T23:20:57.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2026-02-22T23:20:57.330 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:57.331 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:20:57.331 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:20:57.517 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:57.517 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:20:57.527 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-22T23:20:56.168244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81240,"kb_used_data":336,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283034280,"statfs":{"total":289910292480,"available":289827102720,"internally_reserved":0,"allocated":344064,"data_stored":77895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82767187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173445+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:20:54.173445+0000","last_peered":"2026-02-22T23:20:54.173445+0000","last_clean":"2026-02-22T23:20:54.173445+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:20:54.173445+0000","last_undegraded":"2026-02-22T23:20:54.173445+0000","last_fullsized":"2026-02-22T23:20:54.173445+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173310+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:20:54.173310+0000","last_peered":"2026-02-22T23:20:54.173310+0000","last_clean":"2026-02-22T23:20:54.173310+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:20:54.173310+0000","last_undegraded":"2026-02-22T23:20:54.173310+0000","last_fullsized":"2026-02-22T23:20:54.173310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846549+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:20:54.846549+0000","last_peered":"2026-02-22T23:20:54.846549+0000","last_clean":"2026-02-22T23:20:54.846549+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:20:54.846549+0000","last_undegraded":"2026-02-22T23:20:54.846549+0000","last_fullsized":"2026-02-22T23:20:54.846549+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173098+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:20:54.173098+0000","last_peered":"2026-02-22T23:20:54.173098+0000","last_clean":"2026-02-22T23:20:54.173098+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:20:54.173098+0000","last_undegraded":"2026-02-22T23:20:54.173098+0000","last_fullsized":"2026-02-22T23:20:54.173098+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173124+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:20:54.173124+0000","last_peered":"2026-02-22T23:20:54.173124+0000","last_clean":"2026-02-22T23:20:54.173124+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:20:54.173124+0000","last_undegraded":"2026-02-22T23:20:54.173124+0000","last_fullsized":"2026-02-22T23:20:54.173124+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.175091+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:20:54.175091+0000","last_peered":"2026-02-22T23:20:54.175091+0000","last_clean":"2026-02-22T23:20:54.175091+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:20:54.175091+0000","last_undegraded":"2026-02-22T23:20:54.175091+0000","last_fullsized":"2026-02-22T23:20:54.175091+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846521+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:20:54.846521+0000","last_peered":"2026-02-22T23:20:54.846521+0000","last_clean":"2026-02-22T23:20:54.846521+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:20:54.846521+0000","last_undegraded":"2026-02-22T23:20:54.846521+0000","last_fullsized":"2026-02-22T23:20:54.846521+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173265+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:20:54.173265+0000","last_peered":"2026-02-22T23:20:54.173265+0000","last_clean":"2026-02-22T23:20:54.173265+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:20:54.173265+0000","last_undegraded":"2026-02-22T23:20:54.173265+0000","last_fullsized":"2026-02-22T23:20:54.173265+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":114688,"data_stored":26874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":108,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":110592,"data_stored":24128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":118784,"data_stored":26893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:20:57.528 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:20:57.681 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:57.681 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:20:57.690 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-22T23:20:56.168244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81240,"kb_used_data":336,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283034280,"statfs":{"total":289910292480,"available":289827102720,"internally_reserved":0,"allocated":344064,"data_stored":77895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82767187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173445+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:20:54.173445+0000","last_peered":"2026-02-22T23:20:54.173445+0000","last_clean":"2026-02-22T23:20:54.173445+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:20:54.173445+0000","last_undegraded":"2026-02-22T23:20:54.173445+0000","last_fullsized":"2026-02-22T23:20:54.173445+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173310+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:20:54.173310+0000","last_peered":"2026-02-22T23:20:54.173310+0000","last_clean":"2026-02-22T23:20:54.173310+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:20:54.173310+0000","last_undegraded":"2026-02-22T23:20:54.173310+0000","last_fullsized":"2026-02-22T23:20:54.173310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846549+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:20:54.846549+0000","last_peered":"2026-02-22T23:20:54.846549+0000","last_clean":"2026-02-22T23:20:54.846549+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:20:54.846549+0000","last_undegraded":"2026-02-22T23:20:54.846549+0000","last_fullsized":"2026-02-22T23:20:54.846549+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173098+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:20:54.173098+0000","last_peered":"2026-02-22T23:20:54.173098+0000","last_clean":"2026-02-22T23:20:54.173098+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:20:54.173098+0000","last_undegraded":"2026-02-22T23:20:54.173098+0000","last_fullsized":"2026-02-22T23:20:54.173098+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173124+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:20:54.173124+0000","last_peered":"2026-02-22T23:20:54.173124+0000","last_clean":"2026-02-22T23:20:54.173124+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:20:54.173124+0000","last_undegraded":"2026-02-22T23:20:54.173124+0000","last_fullsized":"2026-02-22T23:20:54.173124+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.175091+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:20:54.175091+0000","last_peered":"2026-02-22T23:20:54.175091+0000","last_clean":"2026-02-22T23:20:54.175091+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:20:54.175091+0000","last_undegraded":"2026-02-22T23:20:54.175091+0000","last_fullsized":"2026-02-22T23:20:54.175091+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846521+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:20:54.846521+0000","last_peered":"2026-02-22T23:20:54.846521+0000","last_clean":"2026-02-22T23:20:54.846521+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:20:54.846521+0000","last_undegraded":"2026-02-22T23:20:54.846521+0000","last_fullsized":"2026-02-22T23:20:54.846521+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173265+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:20:54.173265+0000","last_peered":"2026-02-22T23:20:54.173265+0000","last_clean":"2026-02-22T23:20:54.173265+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:20:54.173265+0000","last_undegraded":"2026-02-22T23:20:54.173265+0000","last_fullsized":"2026-02-22T23:20:54.173265+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":114688,"data_stored":26874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":108,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":110592,"data_stored":24128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":118784,"data_stored":26893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:20:57.692 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:20:57.692 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-22T23:20:57.692 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-22T23:20:57.692 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:20:57.857 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:20:57.858 INFO:teuthology.orchestra.run.trial068.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-22T23:20:57.868 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-22T23:20:57.868 INFO:teuthology.run_tasks:Running task divergent_priors... 2026-02-22T23:20:57.887 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-22T23:20:58.039 INFO:teuthology.orchestra.run.trial068.stdout:epoch 13 2026-02-22T23:20:58.039 INFO:teuthology.orchestra.run.trial068.stdout:fsid c2872058-bff0-4756-bb48-18a697ed2e43 2026-02-22T23:20:58.039 INFO:teuthology.orchestra.run.trial068.stdout:created 2026-02-22T23:20:36.088536+0000 2026-02-22T23:20:58.039 INFO:teuthology.orchestra.run.trial068.stdout:modified 2026-02-22T23:20:54.170381+0000 2026-02-22T23:20:58.039 INFO:teuthology.orchestra.run.trial068.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:crush_version 5 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:full_ratio 0.95 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:backfillfull_ratio 0.9 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:nearfull_ratio 0.85 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:require_min_compat_client luminous 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:min_compat_client jewel 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:require_osd_release squid 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:stretch_mode_enabled false 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout: removed_snaps_queue [2~1] 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:max_osd 3 2026-02-22T23:20:58.040 INFO:teuthology.orchestra.run.trial068.stdout:osd.0 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) v2:10.20.193.68:6809/4036091367 v2:10.20.193.68:6810/4036091367 exists,up 6498852d-91da-4351-9faa-b98ee637096f 2026-02-22T23:20:58.041 INFO:teuthology.orchestra.run.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) v2:10.20.193.68:6805/968320453 v2:10.20.193.68:6806/968320453 exists,up 36291856-799f-464f-9583-431e83696b09 2026-02-22T23:20:58.041 INFO:teuthology.orchestra.run.trial068.stdout:osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:10.20.193.68:6801/4010237467 v2:10.20.193.68:6802/4010237467 exists,up c5f32d0e-9755-48db-a210-680579d30eb7 2026-02-22T23:20:58.049 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) v2:10.20.193.68:6809/4036091367 v2:10.20.193.68:6810/4036091367 exists,up 6498852d-91da-4351-9faa-b98ee637096f', 'osd.1 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) v2:10.20.193.68:6805/968320453 v2:10.20.193.68:6806/968320453 exists,up 36291856-799f-464f-9583-431e83696b09', 'osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:10.20.193.68:6801/4010237467 v2:10.20.193.68:6802/4010237467 exists,up c5f32d0e-9755-48db-a210-680579d30eb7'] 2026-02-22T23:20:58.050 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T23:20:58.051 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T23:20:58.051 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T23:20:58.133 INFO:teuthology.orchestra.run.trial068.stdout:34359738373 2026-02-22T23:20:58.134 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:20:58.135 INFO:teuthology.orchestra.run.trial068.stdout:38654705669 2026-02-22T23:20:58.136 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:20:58.137 INFO:teuthology.orchestra.run.trial068.stdout:38654705670 2026-02-22T23:20:58.137 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:20:58.320 INFO:teuthology.orchestra.run.trial068.stdout:34359738373 2026-02-22T23:20:58.323 INFO:teuthology.orchestra.run.trial068.stdout:38654705670 2026-02-22T23:20:58.328 INFO:teuthology.orchestra.run.trial068.stdout:38654705669 2026-02-22T23:20:58.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2026-02-22T23:20:58.331 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:58.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2026-02-22T23:20:58.334 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:58.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2026-02-22T23:20:58.340 DEBUG:teuthology.parallel:result is None 2026-02-22T23:20:58.340 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noout 2026-02-22T23:20:59.181 INFO:teuthology.orchestra.run.trial068.stderr:noout is set 2026-02-22T23:20:59.193 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noin 2026-02-22T23:21:00.182 INFO:teuthology.orchestra.run.trial068.stderr:noin is set 2026-02-22T23:21:00.195 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodown 2026-02-22T23:21:01.044 INFO:teuthology.orchestra.run.trial068.stderr:nodown is set 2026-02-22T23:21:01.059 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:21:01.059 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:01.213 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:01.214 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:01.224 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2026-02-22T23:21:01.044441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81240,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283034280,"statfs":{"total":289910292480,"available":289827102720,"internally_reserved":0,"allocated":352256,"data_stored":80660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82767187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.876197"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173445+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:20:54.173445+0000","last_peered":"2026-02-22T23:20:54.173445+0000","last_clean":"2026-02-22T23:20:54.173445+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:20:54.173445+0000","last_undegraded":"2026-02-22T23:20:54.173445+0000","last_fullsized":"2026-02-22T23:20:54.173445+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173310+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:20:54.173310+0000","last_peered":"2026-02-22T23:20:54.173310+0000","last_clean":"2026-02-22T23:20:54.173310+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:20:54.173310+0000","last_undegraded":"2026-02-22T23:20:54.173310+0000","last_fullsized":"2026-02-22T23:20:54.173310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846549+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:20:54.846549+0000","last_peered":"2026-02-22T23:20:54.846549+0000","last_clean":"2026-02-22T23:20:54.846549+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:20:54.846549+0000","last_undegraded":"2026-02-22T23:20:54.846549+0000","last_fullsized":"2026-02-22T23:20:54.846549+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173098+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:20:54.173098+0000","last_peered":"2026-02-22T23:20:54.173098+0000","last_clean":"2026-02-22T23:20:54.173098+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:20:54.173098+0000","last_undegraded":"2026-02-22T23:20:54.173098+0000","last_fullsized":"2026-02-22T23:20:54.173098+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173124+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:20:54.173124+0000","last_peered":"2026-02-22T23:20:54.173124+0000","last_clean":"2026-02-22T23:20:54.173124+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:20:54.173124+0000","last_undegraded":"2026-02-22T23:20:54.173124+0000","last_fullsized":"2026-02-22T23:20:54.173124+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.175091+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:20:54.175091+0000","last_peered":"2026-02-22T23:20:54.175091+0000","last_clean":"2026-02-22T23:20:54.175091+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:20:54.175091+0000","last_undegraded":"2026-02-22T23:20:54.175091+0000","last_fullsized":"2026-02-22T23:20:54.175091+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846521+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:20:54.846521+0000","last_peered":"2026-02-22T23:20:54.846521+0000","last_clean":"2026-02-22T23:20:54.846521+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:20:54.846521+0000","last_undegraded":"2026-02-22T23:20:54.846521+0000","last_fullsized":"2026-02-22T23:20:54.846521+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173265+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:20:54.173265+0000","last_peered":"2026-02-22T23:20:54.173265+0000","last_clean":"2026-02-22T23:20:54.173265+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:20:54.173265+0000","last_undegraded":"2026-02-22T23:20:54.173265+0000","last_fullsized":"2026-02-22T23:20:54.173265+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":114688,"data_stored":26874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":118784,"data_stored":26893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":26893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:01.225 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:01.372 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:01.373 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:01.383 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2026-02-22T23:21:01.044441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81240,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283034280,"statfs":{"total":289910292480,"available":289827102720,"internally_reserved":0,"allocated":352256,"data_stored":80660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82767187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.876197"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173445+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:20:54.173445+0000","last_peered":"2026-02-22T23:20:54.173445+0000","last_clean":"2026-02-22T23:20:54.173445+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:20:54.173445+0000","last_undegraded":"2026-02-22T23:20:54.173445+0000","last_fullsized":"2026-02-22T23:20:54.173445+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173310+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:20:54.173310+0000","last_peered":"2026-02-22T23:20:54.173310+0000","last_clean":"2026-02-22T23:20:54.173310+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:20:54.173310+0000","last_undegraded":"2026-02-22T23:20:54.173310+0000","last_fullsized":"2026-02-22T23:20:54.173310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846549+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:20:54.846549+0000","last_peered":"2026-02-22T23:20:54.846549+0000","last_clean":"2026-02-22T23:20:54.846549+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:20:54.846549+0000","last_undegraded":"2026-02-22T23:20:54.846549+0000","last_fullsized":"2026-02-22T23:20:54.846549+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173098+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:20:54.173098+0000","last_peered":"2026-02-22T23:20:54.173098+0000","last_clean":"2026-02-22T23:20:54.173098+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:20:54.173098+0000","last_undegraded":"2026-02-22T23:20:54.173098+0000","last_fullsized":"2026-02-22T23:20:54.173098+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173124+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:20:54.173124+0000","last_peered":"2026-02-22T23:20:54.173124+0000","last_clean":"2026-02-22T23:20:54.173124+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:20:54.173124+0000","last_undegraded":"2026-02-22T23:20:54.173124+0000","last_fullsized":"2026-02-22T23:20:54.173124+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.175091+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:20:54.175091+0000","last_peered":"2026-02-22T23:20:54.175091+0000","last_clean":"2026-02-22T23:20:54.175091+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:20:54.175091+0000","last_undegraded":"2026-02-22T23:20:54.175091+0000","last_fullsized":"2026-02-22T23:20:54.175091+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.846521+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:20:54.846521+0000","last_peered":"2026-02-22T23:20:54.846521+0000","last_clean":"2026-02-22T23:20:54.846521+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:20:54.846521+0000","last_undegraded":"2026-02-22T23:20:54.846521+0000","last_fullsized":"2026-02-22T23:20:54.846521+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T23:20:54.173265+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:20:54.173265+0000","last_peered":"2026-02-22T23:20:54.173265+0000","last_clean":"2026-02-22T23:20:54.173265+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:20:54.173265+0000","last_undegraded":"2026-02-22T23:20:54.173265+0000","last_fullsized":"2026-02-22T23:20:54.173265+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":114688,"data_stored":26874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":118784,"data_stored":26893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":26893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:01.385 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:21:01.385 INFO:tasks.divergent_priors:creating foo 2026-02-22T23:21:01.386 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create foo 1 2026-02-22T23:21:02.186 INFO:teuthology.orchestra.run.trial068.stderr:pool 'foo' created 2026-02-22T23:21:02.201 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set osd_min_pg_log_entries 10 2026-02-22T23:21:02.269 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.270 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_min_pg_log_entries = '' (not observed, change may require restart) " 2026-02-22T23:21:02.270 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.277 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set osd_max_pg_log_entries 10 2026-02-22T23:21:02.345 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.346 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_max_pg_log_entries = '' (not observed, change may require restart) " 2026-02-22T23:21:02.346 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.353 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set osd_pg_log_trim_min 5 2026-02-22T23:21:02.423 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.423 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_pg_log_trim_min = '' (not observed, change may require restart) " 2026-02-22T23:21:02.423 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.430 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set osd_min_pg_log_entries 10 2026-02-22T23:21:02.497 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.498 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_min_pg_log_entries = '' (not observed, change may require restart) " 2026-02-22T23:21:02.498 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.505 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set osd_max_pg_log_entries 10 2026-02-22T23:21:02.576 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.576 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_max_pg_log_entries = '' (not observed, change may require restart) " 2026-02-22T23:21:02.576 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.583 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set osd_pg_log_trim_min 5 2026-02-22T23:21:02.644 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.645 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_pg_log_trim_min = '' (not observed, change may require restart) " 2026-02-22T23:21:02.645 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.652 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set osd_min_pg_log_entries 10 2026-02-22T23:21:02.720 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.720 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_min_pg_log_entries = '' (not observed, change may require restart) " 2026-02-22T23:21:02.721 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.728 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set osd_max_pg_log_entries 10 2026-02-22T23:21:02.789 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.789 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_max_pg_log_entries = '' (not observed, change may require restart) " 2026-02-22T23:21:02.789 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.796 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set osd_pg_log_trim_min 5 2026-02-22T23:21:02.865 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:02.865 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_pg_log_trim_min = '' (not observed, change may require restart) " 2026-02-22T23:21:02.865 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:02.872 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:21:03.044 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:03.045 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":17,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:21:02.182548+0000","last_up_change":"2026-02-22T23:20:45.176408+0000","last_in_change":"2026-02-22T23:20:36.705744+0000","flags":"nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800384,"flags_set":["nodown","noin","noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T23:20:50.267994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"foo","create_time":"2026-02-22T23:21:01.544700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":4036091367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":4036091367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6812","nonce":4036091367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6811","nonce":4036091367}]},"public_addr":"10.20.193.68:6809/4036091367","cluster_addr":"10.20.193.68:6810/4036091367","heartbeat_back_addr":"10.20.193.68:6812/4036091367","heartbeat_front_addr":"10.20.193.68:6811/4036091367","state":["exists","up"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":968320453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6806","nonce":968320453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":968320453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":968320453}]},"public_addr":"10.20.193.68:6805/968320453","cluster_addr":"10.20.193.68:6806/968320453","heartbeat_back_addr":"10.20.193.68:6808/968320453","heartbeat_front_addr":"10.20.193.68:6807/968320453","state":["exists","up"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6801","nonce":4010237467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":4010237467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":4010237467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":4010237467}]},"public_addr":"10.20.193.68:6801/4010237467","cluster_addr":"10.20.193.68:6802/4010237467","heartbeat_back_addr":"10.20.193.68:6804/4010237467","heartbeat_front_addr":"10.20.193.68:6803/4010237467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:21:03.055 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg map 2.0 --format=json 2026-02-22T23:21:03.199 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:03.210 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":18,"raw_pgid":"2.0","pgid":"2.0","up":[2,1],"acting":[2,1]} 2026-02-22T23:21:03.210 INFO:tasks.divergent_priors:primary and soon to be divergent is 2 2026-02-22T23:21:03.210 INFO:tasks.divergent_priors:writing initial objects 2026-02-22T23:21:03.211 INFO:tasks.util.rados:rados -p foo put existing_0 /etc/fstab 2026-02-22T23:21:03.211 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_0 /etc/fstab 2026-02-22T23:21:03.979 INFO:tasks.util.rados:rados -p foo put existing_1 /etc/fstab 2026-02-22T23:21:03.979 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_1 /etc/fstab 2026-02-22T23:21:04.005 INFO:tasks.util.rados:rados -p foo put existing_2 /etc/fstab 2026-02-22T23:21:04.005 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_2 /etc/fstab 2026-02-22T23:21:04.072 INFO:tasks.util.rados:rados -p foo put existing_3 /etc/fstab 2026-02-22T23:21:04.072 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_3 /etc/fstab 2026-02-22T23:21:04.149 INFO:tasks.util.rados:rados -p foo put existing_4 /etc/fstab 2026-02-22T23:21:04.149 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_4 /etc/fstab 2026-02-22T23:21:04.216 INFO:tasks.util.rados:rados -p foo put existing_5 /etc/fstab 2026-02-22T23:21:04.216 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_5 /etc/fstab 2026-02-22T23:21:04.288 INFO:tasks.util.rados:rados -p foo put existing_6 /etc/fstab 2026-02-22T23:21:04.288 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_6 /etc/fstab 2026-02-22T23:21:04.355 INFO:tasks.util.rados:rados -p foo put existing_7 /etc/fstab 2026-02-22T23:21:04.355 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_7 /etc/fstab 2026-02-22T23:21:04.420 INFO:tasks.util.rados:rados -p foo put existing_8 /etc/fstab 2026-02-22T23:21:04.420 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_8 /etc/fstab 2026-02-22T23:21:04.487 INFO:tasks.util.rados:rados -p foo put existing_9 /etc/fstab 2026-02-22T23:21:04.488 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_9 /etc/fstab 2026-02-22T23:21:04.559 INFO:tasks.util.rados:rados -p foo put existing_10 /etc/fstab 2026-02-22T23:21:04.559 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_10 /etc/fstab 2026-02-22T23:21:04.630 INFO:tasks.util.rados:rados -p foo put existing_11 /etc/fstab 2026-02-22T23:21:04.631 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_11 /etc/fstab 2026-02-22T23:21:04.698 INFO:tasks.util.rados:rados -p foo put existing_12 /etc/fstab 2026-02-22T23:21:04.699 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_12 /etc/fstab 2026-02-22T23:21:04.763 INFO:tasks.util.rados:rados -p foo put existing_13 /etc/fstab 2026-02-22T23:21:04.764 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_13 /etc/fstab 2026-02-22T23:21:04.831 INFO:tasks.util.rados:rados -p foo put existing_14 /etc/fstab 2026-02-22T23:21:04.831 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_14 /etc/fstab 2026-02-22T23:21:04.895 INFO:tasks.util.rados:rados -p foo put existing_15 /etc/fstab 2026-02-22T23:21:04.895 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_15 /etc/fstab 2026-02-22T23:21:04.963 INFO:tasks.util.rados:rados -p foo put existing_16 /etc/fstab 2026-02-22T23:21:04.963 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_16 /etc/fstab 2026-02-22T23:21:05.038 INFO:tasks.util.rados:rados -p foo put existing_17 /etc/fstab 2026-02-22T23:21:05.039 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_17 /etc/fstab 2026-02-22T23:21:05.103 INFO:tasks.util.rados:rados -p foo put existing_18 /etc/fstab 2026-02-22T23:21:05.104 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_18 /etc/fstab 2026-02-22T23:21:05.170 INFO:tasks.util.rados:rados -p foo put existing_19 /etc/fstab 2026-02-22T23:21:05.170 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_19 /etc/fstab 2026-02-22T23:21:05.235 INFO:tasks.util.rados:rados -p foo put existing_20 /etc/fstab 2026-02-22T23:21:05.235 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_20 /etc/fstab 2026-02-22T23:21:05.302 INFO:tasks.util.rados:rados -p foo put existing_21 /etc/fstab 2026-02-22T23:21:05.303 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_21 /etc/fstab 2026-02-22T23:21:05.370 INFO:tasks.util.rados:rados -p foo put existing_22 /etc/fstab 2026-02-22T23:21:05.371 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_22 /etc/fstab 2026-02-22T23:21:05.435 INFO:tasks.util.rados:rados -p foo put existing_23 /etc/fstab 2026-02-22T23:21:05.436 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_23 /etc/fstab 2026-02-22T23:21:05.506 INFO:tasks.util.rados:rados -p foo put existing_24 /etc/fstab 2026-02-22T23:21:05.506 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_24 /etc/fstab 2026-02-22T23:21:05.575 INFO:tasks.util.rados:rados -p foo put existing_25 /etc/fstab 2026-02-22T23:21:05.575 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_25 /etc/fstab 2026-02-22T23:21:05.647 INFO:tasks.util.rados:rados -p foo put existing_26 /etc/fstab 2026-02-22T23:21:05.647 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_26 /etc/fstab 2026-02-22T23:21:05.711 INFO:tasks.util.rados:rados -p foo put existing_27 /etc/fstab 2026-02-22T23:21:05.711 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_27 /etc/fstab 2026-02-22T23:21:05.782 INFO:tasks.util.rados:rados -p foo put existing_28 /etc/fstab 2026-02-22T23:21:05.782 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_28 /etc/fstab 2026-02-22T23:21:05.846 INFO:tasks.util.rados:rados -p foo put existing_29 /etc/fstab 2026-02-22T23:21:05.846 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_29 /etc/fstab 2026-02-22T23:21:05.910 INFO:tasks.util.rados:rados -p foo put existing_30 /etc/fstab 2026-02-22T23:21:05.910 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_30 /etc/fstab 2026-02-22T23:21:05.976 INFO:tasks.util.rados:rados -p foo put existing_31 /etc/fstab 2026-02-22T23:21:05.977 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_31 /etc/fstab 2026-02-22T23:21:06.047 INFO:tasks.util.rados:rados -p foo put existing_32 /etc/fstab 2026-02-22T23:21:06.047 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_32 /etc/fstab 2026-02-22T23:21:06.111 INFO:tasks.util.rados:rados -p foo put existing_33 /etc/fstab 2026-02-22T23:21:06.111 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_33 /etc/fstab 2026-02-22T23:21:06.178 INFO:tasks.util.rados:rados -p foo put existing_34 /etc/fstab 2026-02-22T23:21:06.178 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_34 /etc/fstab 2026-02-22T23:21:06.243 INFO:tasks.util.rados:rados -p foo put existing_35 /etc/fstab 2026-02-22T23:21:06.244 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_35 /etc/fstab 2026-02-22T23:21:06.311 INFO:tasks.util.rados:rados -p foo put existing_36 /etc/fstab 2026-02-22T23:21:06.312 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_36 /etc/fstab 2026-02-22T23:21:06.380 INFO:tasks.util.rados:rados -p foo put existing_37 /etc/fstab 2026-02-22T23:21:06.380 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_37 /etc/fstab 2026-02-22T23:21:06.451 INFO:tasks.util.rados:rados -p foo put existing_38 /etc/fstab 2026-02-22T23:21:06.451 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_38 /etc/fstab 2026-02-22T23:21:06.519 INFO:tasks.util.rados:rados -p foo put existing_39 /etc/fstab 2026-02-22T23:21:06.519 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_39 /etc/fstab 2026-02-22T23:21:06.584 INFO:tasks.util.rados:rados -p foo put existing_40 /etc/fstab 2026-02-22T23:21:06.585 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_40 /etc/fstab 2026-02-22T23:21:06.652 INFO:tasks.util.rados:rados -p foo put existing_41 /etc/fstab 2026-02-22T23:21:06.653 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_41 /etc/fstab 2026-02-22T23:21:06.718 INFO:tasks.util.rados:rados -p foo put existing_42 /etc/fstab 2026-02-22T23:21:06.718 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_42 /etc/fstab 2026-02-22T23:21:06.787 INFO:tasks.util.rados:rados -p foo put existing_43 /etc/fstab 2026-02-22T23:21:06.787 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_43 /etc/fstab 2026-02-22T23:21:06.851 INFO:tasks.util.rados:rados -p foo put existing_44 /etc/fstab 2026-02-22T23:21:06.851 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_44 /etc/fstab 2026-02-22T23:21:06.931 INFO:tasks.util.rados:rados -p foo put existing_45 /etc/fstab 2026-02-22T23:21:06.931 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_45 /etc/fstab 2026-02-22T23:21:07.003 INFO:tasks.util.rados:rados -p foo put existing_46 /etc/fstab 2026-02-22T23:21:07.004 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_46 /etc/fstab 2026-02-22T23:21:07.071 INFO:tasks.util.rados:rados -p foo put existing_47 /etc/fstab 2026-02-22T23:21:07.071 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_47 /etc/fstab 2026-02-22T23:21:07.139 INFO:tasks.util.rados:rados -p foo put existing_48 /etc/fstab 2026-02-22T23:21:07.140 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_48 /etc/fstab 2026-02-22T23:21:07.206 INFO:tasks.util.rados:rados -p foo put existing_49 /etc/fstab 2026-02-22T23:21:07.206 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_49 /etc/fstab 2026-02-22T23:21:07.275 INFO:tasks.util.rados:rados -p foo put existing_50 /etc/fstab 2026-02-22T23:21:07.275 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_50 /etc/fstab 2026-02-22T23:21:07.343 INFO:tasks.util.rados:rados -p foo put existing_51 /etc/fstab 2026-02-22T23:21:07.343 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_51 /etc/fstab 2026-02-22T23:21:07.408 INFO:tasks.util.rados:rados -p foo put existing_52 /etc/fstab 2026-02-22T23:21:07.408 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_52 /etc/fstab 2026-02-22T23:21:07.476 INFO:tasks.util.rados:rados -p foo put existing_53 /etc/fstab 2026-02-22T23:21:07.476 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_53 /etc/fstab 2026-02-22T23:21:07.543 INFO:tasks.util.rados:rados -p foo put existing_54 /etc/fstab 2026-02-22T23:21:07.543 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_54 /etc/fstab 2026-02-22T23:21:07.606 INFO:tasks.util.rados:rados -p foo put existing_55 /etc/fstab 2026-02-22T23:21:07.606 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_55 /etc/fstab 2026-02-22T23:21:07.670 INFO:tasks.util.rados:rados -p foo put existing_56 /etc/fstab 2026-02-22T23:21:07.670 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_56 /etc/fstab 2026-02-22T23:21:07.739 INFO:tasks.util.rados:rados -p foo put existing_57 /etc/fstab 2026-02-22T23:21:07.739 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_57 /etc/fstab 2026-02-22T23:21:07.803 INFO:tasks.util.rados:rados -p foo put existing_58 /etc/fstab 2026-02-22T23:21:07.804 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_58 /etc/fstab 2026-02-22T23:21:07.888 INFO:tasks.util.rados:rados -p foo put existing_59 /etc/fstab 2026-02-22T23:21:07.888 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_59 /etc/fstab 2026-02-22T23:21:07.960 INFO:tasks.util.rados:rados -p foo put existing_60 /etc/fstab 2026-02-22T23:21:07.960 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_60 /etc/fstab 2026-02-22T23:21:08.032 INFO:tasks.util.rados:rados -p foo put existing_61 /etc/fstab 2026-02-22T23:21:08.032 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_61 /etc/fstab 2026-02-22T23:21:08.099 INFO:tasks.util.rados:rados -p foo put existing_62 /etc/fstab 2026-02-22T23:21:08.099 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_62 /etc/fstab 2026-02-22T23:21:08.167 INFO:tasks.util.rados:rados -p foo put existing_63 /etc/fstab 2026-02-22T23:21:08.167 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_63 /etc/fstab 2026-02-22T23:21:08.231 INFO:tasks.util.rados:rados -p foo put existing_64 /etc/fstab 2026-02-22T23:21:08.231 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_64 /etc/fstab 2026-02-22T23:21:08.295 INFO:tasks.util.rados:rados -p foo put existing_65 /etc/fstab 2026-02-22T23:21:08.295 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_65 /etc/fstab 2026-02-22T23:21:08.359 INFO:tasks.util.rados:rados -p foo put existing_66 /etc/fstab 2026-02-22T23:21:08.359 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_66 /etc/fstab 2026-02-22T23:21:08.423 INFO:tasks.util.rados:rados -p foo put existing_67 /etc/fstab 2026-02-22T23:21:08.424 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_67 /etc/fstab 2026-02-22T23:21:08.491 INFO:tasks.util.rados:rados -p foo put existing_68 /etc/fstab 2026-02-22T23:21:08.491 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_68 /etc/fstab 2026-02-22T23:21:08.559 INFO:tasks.util.rados:rados -p foo put existing_69 /etc/fstab 2026-02-22T23:21:08.560 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_69 /etc/fstab 2026-02-22T23:21:08.631 INFO:tasks.util.rados:rados -p foo put existing_70 /etc/fstab 2026-02-22T23:21:08.631 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_70 /etc/fstab 2026-02-22T23:21:08.698 INFO:tasks.util.rados:rados -p foo put existing_71 /etc/fstab 2026-02-22T23:21:08.698 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_71 /etc/fstab 2026-02-22T23:21:08.763 INFO:tasks.util.rados:rados -p foo put existing_72 /etc/fstab 2026-02-22T23:21:08.764 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_72 /etc/fstab 2026-02-22T23:21:08.830 INFO:tasks.util.rados:rados -p foo put existing_73 /etc/fstab 2026-02-22T23:21:08.830 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_73 /etc/fstab 2026-02-22T23:21:08.895 INFO:tasks.util.rados:rados -p foo put existing_74 /etc/fstab 2026-02-22T23:21:08.896 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_74 /etc/fstab 2026-02-22T23:21:08.966 INFO:tasks.util.rados:rados -p foo put existing_75 /etc/fstab 2026-02-22T23:21:08.967 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_75 /etc/fstab 2026-02-22T23:21:09.031 INFO:tasks.util.rados:rados -p foo put existing_76 /etc/fstab 2026-02-22T23:21:09.031 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_76 /etc/fstab 2026-02-22T23:21:09.095 INFO:tasks.util.rados:rados -p foo put existing_77 /etc/fstab 2026-02-22T23:21:09.095 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_77 /etc/fstab 2026-02-22T23:21:09.163 INFO:tasks.util.rados:rados -p foo put existing_78 /etc/fstab 2026-02-22T23:21:09.163 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_78 /etc/fstab 2026-02-22T23:21:09.226 INFO:tasks.util.rados:rados -p foo put existing_79 /etc/fstab 2026-02-22T23:21:09.226 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_79 /etc/fstab 2026-02-22T23:21:09.291 INFO:tasks.util.rados:rados -p foo put existing_80 /etc/fstab 2026-02-22T23:21:09.292 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_80 /etc/fstab 2026-02-22T23:21:09.359 INFO:tasks.util.rados:rados -p foo put existing_81 /etc/fstab 2026-02-22T23:21:09.359 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_81 /etc/fstab 2026-02-22T23:21:09.424 INFO:tasks.util.rados:rados -p foo put existing_82 /etc/fstab 2026-02-22T23:21:09.424 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_82 /etc/fstab 2026-02-22T23:21:09.506 INFO:tasks.util.rados:rados -p foo put existing_83 /etc/fstab 2026-02-22T23:21:09.506 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_83 /etc/fstab 2026-02-22T23:21:09.581 INFO:tasks.util.rados:rados -p foo put existing_84 /etc/fstab 2026-02-22T23:21:09.581 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_84 /etc/fstab 2026-02-22T23:21:09.647 INFO:tasks.util.rados:rados -p foo put existing_85 /etc/fstab 2026-02-22T23:21:09.647 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_85 /etc/fstab 2026-02-22T23:21:09.712 INFO:tasks.util.rados:rados -p foo put existing_86 /etc/fstab 2026-02-22T23:21:09.712 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_86 /etc/fstab 2026-02-22T23:21:09.780 INFO:tasks.util.rados:rados -p foo put existing_87 /etc/fstab 2026-02-22T23:21:09.780 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_87 /etc/fstab 2026-02-22T23:21:09.848 INFO:tasks.util.rados:rados -p foo put existing_88 /etc/fstab 2026-02-22T23:21:09.848 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_88 /etc/fstab 2026-02-22T23:21:09.915 INFO:tasks.util.rados:rados -p foo put existing_89 /etc/fstab 2026-02-22T23:21:09.915 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_89 /etc/fstab 2026-02-22T23:21:09.978 INFO:tasks.util.rados:rados -p foo put existing_90 /etc/fstab 2026-02-22T23:21:09.978 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_90 /etc/fstab 2026-02-22T23:21:10.043 INFO:tasks.util.rados:rados -p foo put existing_91 /etc/fstab 2026-02-22T23:21:10.044 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_91 /etc/fstab 2026-02-22T23:21:10.112 INFO:tasks.util.rados:rados -p foo put existing_92 /etc/fstab 2026-02-22T23:21:10.112 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_92 /etc/fstab 2026-02-22T23:21:10.179 INFO:tasks.util.rados:rados -p foo put existing_93 /etc/fstab 2026-02-22T23:21:10.179 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_93 /etc/fstab 2026-02-22T23:21:10.243 INFO:tasks.util.rados:rados -p foo put existing_94 /etc/fstab 2026-02-22T23:21:10.244 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_94 /etc/fstab 2026-02-22T23:21:10.313 INFO:tasks.util.rados:rados -p foo put existing_95 /etc/fstab 2026-02-22T23:21:10.313 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_95 /etc/fstab 2026-02-22T23:21:10.379 INFO:tasks.util.rados:rados -p foo put existing_96 /etc/fstab 2026-02-22T23:21:10.379 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_96 /etc/fstab 2026-02-22T23:21:10.447 INFO:tasks.util.rados:rados -p foo put existing_97 /etc/fstab 2026-02-22T23:21:10.447 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_97 /etc/fstab 2026-02-22T23:21:10.511 INFO:tasks.util.rados:rados -p foo put existing_98 /etc/fstab 2026-02-22T23:21:10.511 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_98 /etc/fstab 2026-02-22T23:21:10.580 INFO:tasks.util.rados:rados -p foo put existing_99 /etc/fstab 2026-02-22T23:21:10.581 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_99 /etc/fstab 2026-02-22T23:21:10.648 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:21:10.648 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:10.834 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:10.835 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:10.844 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-22T23:21:10.170024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13851,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81576,"kb_used_data":676,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283033944,"statfs":{"total":289910292480,"available":289826758656,"internally_reserved":0,"allocated":692224,"data_stored":162357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82767187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13832,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.983613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:04.189895+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:04.189909+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:04.213382+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:04.189904+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'19","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:05.168057+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:05.168057+0000","last_peered":"2026-02-22T23:21:05.168057+0000","last_clean":"2026-02-22T23:21:05.168057+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:05.168057+0000","last_undegraded":"2026-02-22T23:21:05.168057+0000","last_fullsized":"2026-02-22T23:21:05.168057+0000","mapping_epoch":17,"log_start":"19'5","ondisk_log_start":"19'5","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":5,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:04.189960+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:04.808232+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:04.213379+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:04.189880+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":13832,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":38584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":229376,"data_stored":54349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":294912,"data_stored":65288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":167936,"data_stored":42720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":24752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":13832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:10.846 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:10.998 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:10.998 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:11.007 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-22T23:21:10.170024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13851,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81576,"kb_used_data":676,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283033944,"statfs":{"total":289910292480,"available":289826758656,"internally_reserved":0,"allocated":692224,"data_stored":162357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82767187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13832,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.983613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:04.189895+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:04.189909+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:04.213382+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:04.189904+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'19","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:05.168057+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:05.168057+0000","last_peered":"2026-02-22T23:21:05.168057+0000","last_clean":"2026-02-22T23:21:05.168057+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:05.168057+0000","last_undegraded":"2026-02-22T23:21:05.168057+0000","last_fullsized":"2026-02-22T23:21:05.168057+0000","mapping_epoch":17,"log_start":"19'5","ondisk_log_start":"19'5","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":5,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:04.189960+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:04.808232+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:04.213379+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:04.189880+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":13832,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":38584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":229376,"data_stored":54349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":294912,"data_stored":65288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":167936,"data_stored":42720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":24752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":13832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:11.009 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:21:11.009 INFO:tasks.divergent_priors:blackholing osds [0, 1] 2026-02-22T23:21:11.010 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-22T23:21:11.079 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:11.079 INFO:teuthology.orchestra.run.trial068.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:21:11.079 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:11.086 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set objectstore_blackhole 1 2026-02-22T23:21:11.156 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:11.156 INFO:teuthology.orchestra.run.trial068.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:21:11.157 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:11.163 INFO:tasks.divergent_priors:writing divergent objects 2026-02-22T23:21:11.163 INFO:tasks.util.rados:rados -p foo put existing_0 /etc/resolv.conf 2026-02-22T23:21:11.164 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_0 /etc/resolv.conf 2026-02-22T23:21:11.166 INFO:tasks.util.rados:rados -p foo put existing_1 /etc/resolv.conf 2026-02-22T23:21:11.166 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_1 /etc/resolv.conf 2026-02-22T23:21:11.186 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.181+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.187 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.181+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.187 INFO:tasks.util.rados:rados -p foo put existing_2 /etc/resolv.conf 2026-02-22T23:21:11.187 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_2 /etc/resolv.conf 2026-02-22T23:21:11.208 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.205+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.208 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.205+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.208 INFO:tasks.util.rados:rados -p foo put existing_3 /etc/resolv.conf 2026-02-22T23:21:11.208 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_3 /etc/resolv.conf 2026-02-22T23:21:11.210 INFO:tasks.util.rados:rados -p foo put existing_4 /etc/resolv.conf 2026-02-22T23:21:11.211 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_4 /etc/resolv.conf 2026-02-22T23:21:11.231 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.229+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.231 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.229+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.232 INFO:tasks.divergent_priors:remove divergent objects 2026-02-22T23:21:11.232 INFO:tasks.util.rados:rados -p foo rm existing_5 2026-02-22T23:21:11.232 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo rm existing_5 2026-02-22T23:21:11.234 INFO:tasks.util.rados:rados -p foo rm existing_6 2026-02-22T23:21:11.234 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo rm existing_6 2026-02-22T23:21:11.234 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.229+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.234 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.229+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.255 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.249+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.255 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.249+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.255 INFO:tasks.util.rados:rados -p foo rm existing_7 2026-02-22T23:21:11.255 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo rm existing_7 2026-02-22T23:21:11.257 INFO:tasks.util.rados:rados -p foo rm existing_8 2026-02-22T23:21:11.257 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo rm existing_8 2026-02-22T23:21:11.260 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.257+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.260 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.257+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.261 INFO:tasks.util.rados:rados -p foo rm existing_9 2026-02-22T23:21:11.261 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo rm existing_9 2026-02-22T23:21:11.275 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.269+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.275 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.269+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.276 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.273+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.276 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.273+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.280 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.277+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.280 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.277+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.284 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.281+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:11.284 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:11.281+0000 7f3608f78640 -1 bdev(0x557ec3ccac00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:12.840 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:12.837+0000 7f360cf80640 -1 bdev(0x557ec2d5dc00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:12.840 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:12.837+0000 7f360cf80640 -1 bdev(0x557ec2d5dc00 /var/lib/ceph/osd/ceph-1/block) aio_write objectstore_blackhole=true, throwing out IO 2026-02-22T23:21:21.265 DEBUG:teuthology.orchestra.run.trial068:> killall -9 rados 2026-02-22T23:21:21.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:21:21.274 INFO:teuthology.orchestra.run.trial068.stderr:rados: no process found 2026-02-22T23:21:21.275 INFO:tasks.divergent_priors:killing all the osds 2026-02-22T23:21:21.275 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-22T23:21:21.275 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:21:22.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-22T23:21:27.376 INFO:tasks.ceph.osd.0:Stopped 2026-02-22T23:21:27.377 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-22T23:21:27.377 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:21:27.450 INFO:tasks.ceph.osd.1:Stopped 2026-02-22T23:21:27.451 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T23:21:27.451 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:21:27.506 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2026-02-22T23:21:27.519 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T23:21:27.519 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2026-02-22T23:21:27.670 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:27.665+0000 7f270bc3a640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2026-02-22T23:21:27.721 INFO:teuthology.orchestra.run.trial068.stderr:marked down osd.0. 2026-02-22T23:21:27.735 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2026-02-22T23:21:27.884 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:27.881+0000 7f270bc3a640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2026-02-22T23:21:28.727 INFO:teuthology.orchestra.run.trial068.stderr:marked down osd.1. 2026-02-22T23:21:28.741 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2026-02-22T23:21:28.898 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:28.893+0000 7f270bc3a640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2026-02-22T23:21:29.731 INFO:teuthology.orchestra.run.trial068.stderr:marked down osd.2. 2026-02-22T23:21:29.744 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2026-02-22T23:21:29.907 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:29.901+0000 7f270bc3a640 -1 mon.a@0(leader).osd e22 definitely_dead 0 2026-02-22T23:21:30.733 INFO:teuthology.orchestra.run.trial068.stderr:marked out osd.0. 2026-02-22T23:21:30.745 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2026-02-22T23:21:30.906 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:30.901+0000 7f270bc3a640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2026-02-22T23:21:31.735 INFO:teuthology.orchestra.run.trial068.stderr:marked out osd.1. 2026-02-22T23:21:31.748 INFO:tasks.divergent_priors:bringing up non_divergent [0, 1] 2026-02-22T23:21:31.748 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-22T23:21:31.748 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-22T23:21:31.749 INFO:tasks.ceph.osd.0:Started 2026-02-22T23:21:31.749 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2026-02-22T23:21:31.807 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:21:31.807 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:21:31.807 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:21:31.807 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:config diff 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:config diff get 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:config get 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:config help [] 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:config set ... 2026-02-22T23:21:31.808 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:21:31.810 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:21:31.810 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2026-02-22T23:21:32.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-22T23:21:33.025 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:21:33.021+0000 7f08de2cb8c0 -1 Falling back to public interface 2026-02-22T23:21:36.811 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2026-02-22T23:21:36.860 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:21:36.857+0000 7f08de2cb8c0 -1 osd.0 19 log_to_monitors true 2026-02-22T23:21:36.880 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:21:36.880 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:21:36.880 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:21:36.880 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:21:36.880 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:bluefs debug_inject_read_zeros 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:bluefs files list 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:bluefs stats 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:bluestore allocator dump block 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:bluestore allocator fragmentation block 2026-02-22T23:21:36.881 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:21:36.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:21:36.883 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2026-02-22T23:21:37.243 INFO:tasks.ceph.osd.0.trial068.stderr:2026-02-22T23:21:37.237+0000 7f08d60af640 -1 osd.0 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:21:38.195 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:21:38.189+0000 7fb569176640 -1 calc_pg_upmaps abort due to max <= 0 2026-02-22T23:21:38.195 INFO:tasks.ceph.mgr.x.trial068.stderr:2026-02-22T23:21:38.189+0000 7fb569176640 -1 calc_pg_upmaps abort due to max <= 0 2026-02-22T23:21:41.884 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial068.stdout: "ops": [], 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial068.stdout: "num_ops": 0 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:41.966 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-22T23:21:41.967 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-22T23:21:41.969 INFO:tasks.ceph.osd.1:Started 2026-02-22T23:21:41.969 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-22T23:21:42.027 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:config diff 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:config diff get 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:config get 2026-02-22T23:21:42.028 INFO:teuthology.orchestra.run.trial068.stderr:config help [] 2026-02-22T23:21:42.029 INFO:teuthology.orchestra.run.trial068.stderr:config set ... 2026-02-22T23:21:42.029 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:21:42.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:21:42.030 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-22T23:21:43.217 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:43.213+0000 7fef197738c0 -1 Falling back to public interface 2026-02-22T23:21:46.994 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:46.989+0000 7fef197738c0 -1 osd.1 19 log_to_monitors true 2026-02-22T23:21:47.030 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-22T23:21:47.088 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:bluefs debug_inject_read_zeros 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:bluefs files list 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:bluefs stats 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:bluestore allocator dump block 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:bluestore allocator fragmentation block 2026-02-22T23:21:47.089 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:21:47.090 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:21:47.091 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-22T23:21:47.242 INFO:tasks.ceph.osd.1.trial068.stderr:2026-02-22T23:21:47.237+0000 7fef11557640 -1 osd.1 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:21:52.092 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-22T23:21:52.164 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:21:52.164 INFO:teuthology.orchestra.run.trial068.stdout: "ops": [], 2026-02-22T23:21:52.164 INFO:teuthology.orchestra.run.trial068.stdout: "num_ops": 0 2026-02-22T23:21:52.165 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:21:52.172 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2026-02-22T23:21:52.322 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:52.317+0000 7f270bc3a640 -1 mon.a@0(leader).osd e28 definitely_dead 0 2026-02-22T23:21:53.243 INFO:teuthology.orchestra.run.trial068.stderr:marked in osd.0. 2026-02-22T23:21:53.257 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2026-02-22T23:21:53.411 INFO:tasks.ceph.mon.a.trial068.stderr:2026-02-22T23:21:53.405+0000 7f270bc3a640 -1 mon.a@0(leader).osd e29 definitely_dead 0 2026-02-22T23:21:54.246 INFO:teuthology.orchestra.run.trial068.stderr:marked in osd.1. 2026-02-22T23:21:54.259 INFO:tasks.divergent_priors:writing non-divergent object existing_10 2026-02-22T23:21:54.259 INFO:tasks.util.rados:rados -p foo put existing_10 /etc/resolv.conf 2026-02-22T23:21:54.259 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put existing_10 /etc/resolv.conf 2026-02-22T23:21:55.256 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2026-02-22T23:21:55.256 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:55.400 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:55.400 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:55.410 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2026-02-22T23:21:55.246028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70374,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55232,"kb_used_data":776,"kb_used_omap":5,"kb_used_meta":54394,"kb_avail":188688448,"statfs":{"total":193273528320,"available":193216970752,"internally_reserved":0,"allocated":794624,"data_stored":182134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6028,"internal_metadata":55699572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.005807"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:28.726687+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:28.726690+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:29.730320+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:28.726691+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'110","reported_seq":122,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:11.280397+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:11.280397+0000","last_peered":"2026-02-22T23:21:11.280397+0000","last_clean":"2026-02-22T23:21:11.280397+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:29.730321+0000","last_undegraded":"2026-02-22T23:21:11.280397+0000","last_fullsized":"2026-02-22T23:21:11.280397+0000","mapping_epoch":17,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":95,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:28.726693+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:27.720945+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:29.730323+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:28.726694+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":143155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738379,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":552960,"data_stored":113056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":120259084291,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":236,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":241664,"data_stored":69078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":72800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":70355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:55.411 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:55.557 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:55.558 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:55.567 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2026-02-22T23:21:55.246028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70374,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55232,"kb_used_data":776,"kb_used_omap":5,"kb_used_meta":54394,"kb_avail":188688448,"statfs":{"total":193273528320,"available":193216970752,"internally_reserved":0,"allocated":794624,"data_stored":182134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6028,"internal_metadata":55699572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.005807"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:28.726687+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:28.726690+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:29.730320+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:28.726691+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'110","reported_seq":122,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:11.280397+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:11.280397+0000","last_peered":"2026-02-22T23:21:11.280397+0000","last_clean":"2026-02-22T23:21:11.280397+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:29.730321+0000","last_undegraded":"2026-02-22T23:21:11.280397+0000","last_fullsized":"2026-02-22T23:21:11.280397+0000","mapping_epoch":17,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":95,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:28.726693+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:27.720945+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:29.730323+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:28.726694+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":143155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738379,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":552960,"data_stored":113056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":120259084291,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":236,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":241664,"data_stored":69078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":72800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":70355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:55.568 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:55.720 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:55.721 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:55.730 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2026-02-22T23:21:55.246028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70374,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55232,"kb_used_data":776,"kb_used_omap":5,"kb_used_meta":54394,"kb_avail":188688448,"statfs":{"total":193273528320,"available":193216970752,"internally_reserved":0,"allocated":794624,"data_stored":182134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6028,"internal_metadata":55699572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.005807"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:28.726687+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:28.726690+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:29.730320+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:28.726691+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'110","reported_seq":122,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:11.280397+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:11.280397+0000","last_peered":"2026-02-22T23:21:11.280397+0000","last_clean":"2026-02-22T23:21:11.280397+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:29.730321+0000","last_undegraded":"2026-02-22T23:21:11.280397+0000","last_fullsized":"2026-02-22T23:21:11.280397+0000","mapping_epoch":17,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":95,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:28.726693+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:27.720945+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:29.730323+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:28.726694+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":143155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738379,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":552960,"data_stored":113056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":120259084291,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":236,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":241664,"data_stored":69078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":72800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":70355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:58.732 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:58.885 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:58.886 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:58.895 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2026-02-22T23:21:58.175436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70374,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55232,"kb_used_data":776,"kb_used_omap":5,"kb_used_meta":54394,"kb_avail":188688448,"statfs":{"total":193273528320,"available":193216970752,"internally_reserved":0,"allocated":794624,"data_stored":182134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6028,"internal_metadata":55699572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.933366"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:28.726687+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:28.726690+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:29.730320+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:28.726691+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'110","reported_seq":122,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:11.280397+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:11.280397+0000","last_peered":"2026-02-22T23:21:11.280397+0000","last_clean":"2026-02-22T23:21:11.280397+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:29.730321+0000","last_undegraded":"2026-02-22T23:21:11.280397+0000","last_fullsized":"2026-02-22T23:21:11.280397+0000","mapping_epoch":17,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":95,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:28.726693+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:27.720945+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:29.730323+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:28.726694+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":143155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738379,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":552960,"data_stored":113056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":120259084291,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":236,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":241664,"data_stored":69078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":72800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":70355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:21:58.897 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:21:59.048 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:21:59.048 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:21:59.057 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2026-02-22T23:21:58.175436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70374,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55232,"kb_used_data":776,"kb_used_omap":5,"kb_used_meta":54394,"kb_avail":188688448,"statfs":{"total":193273528320,"available":193216970752,"internally_reserved":0,"allocated":794624,"data_stored":182134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6028,"internal_metadata":55699572},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.933366"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189895+0000","last_change":"2026-02-22T23:20:54.173576+0000","last_active":"2026-02-22T23:21:04.189895+0000","last_peered":"2026-02-22T23:21:04.189895+0000","last_clean":"2026-02-22T23:21:04.189895+0000","last_became_active":"2026-02-22T23:20:52.170503+0000","last_became_peered":"2026-02-22T23:20:52.170503+0000","last_unstale":"2026-02-22T23:21:28.726687+0000","last_undegraded":"2026-02-22T23:21:04.189895+0000","last_fullsized":"2026-02-22T23:21:04.189895+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:24:40.877250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189909+0000","last_change":"2026-02-22T23:20:54.173416+0000","last_active":"2026-02-22T23:21:04.189909+0000","last_peered":"2026-02-22T23:21:04.189909+0000","last_clean":"2026-02-22T23:21:04.189909+0000","last_became_active":"2026-02-22T23:20:52.170724+0000","last_became_peered":"2026-02-22T23:20:52.170724+0000","last_unstale":"2026-02-22T23:21:28.726690+0000","last_undegraded":"2026-02-22T23:21:04.189909+0000","last_fullsized":"2026-02-22T23:21:04.189909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:19.090804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213382+0000","last_change":"2026-02-22T23:20:54.846725+0000","last_active":"2026-02-22T23:21:04.213382+0000","last_peered":"2026-02-22T23:21:04.213382+0000","last_clean":"2026-02-22T23:21:04.213382+0000","last_became_active":"2026-02-22T23:20:52.171260+0000","last_became_peered":"2026-02-22T23:20:52.171260+0000","last_unstale":"2026-02-22T23:21:29.730320+0000","last_undegraded":"2026-02-22T23:21:04.213382+0000","last_fullsized":"2026-02-22T23:21:04.213382+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:10:12.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029373700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189904+0000","last_change":"2026-02-22T23:20:54.173227+0000","last_active":"2026-02-22T23:21:04.189904+0000","last_peered":"2026-02-22T23:21:04.189904+0000","last_clean":"2026-02-22T23:21:04.189904+0000","last_became_active":"2026-02-22T23:20:52.171042+0000","last_became_peered":"2026-02-22T23:20:52.171042+0000","last_unstale":"2026-02-22T23:21:28.726691+0000","last_undegraded":"2026-02-22T23:21:04.189904+0000","last_fullsized":"2026-02-22T23:21:04.189904+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:20:13.242661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"19'110","reported_seq":122,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:11.280397+0000","last_change":"2026-02-22T23:21:03.974064+0000","last_active":"2026-02-22T23:21:11.280397+0000","last_peered":"2026-02-22T23:21:11.280397+0000","last_clean":"2026-02-22T23:21:11.280397+0000","last_became_active":"2026-02-22T23:21:03.973375+0000","last_became_peered":"2026-02-22T23:21:03.973375+0000","last_unstale":"2026-02-22T23:21:29.730321+0000","last_undegraded":"2026-02-22T23:21:11.280397+0000","last_fullsized":"2026-02-22T23:21:11.280397+0000","mapping_epoch":17,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":95,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:03:55.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":32,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189960+0000","last_change":"2026-02-22T23:20:54.173240+0000","last_active":"2026-02-22T23:21:04.189960+0000","last_peered":"2026-02-22T23:21:04.189960+0000","last_clean":"2026-02-22T23:21:04.189960+0000","last_became_active":"2026-02-22T23:20:52.170508+0000","last_became_peered":"2026-02-22T23:20:52.170508+0000","last_unstale":"2026-02-22T23:21:28.726693+0000","last_undegraded":"2026-02-22T23:21:04.189960+0000","last_fullsized":"2026-02-22T23:21:04.189960+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:05.285589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.808232+0000","last_change":"2026-02-22T23:20:54.175091+0000","last_active":"2026-02-22T23:21:04.808232+0000","last_peered":"2026-02-22T23:21:04.808232+0000","last_clean":"2026-02-22T23:21:04.808232+0000","last_became_active":"2026-02-22T23:20:52.170721+0000","last_became_peered":"2026-02-22T23:20:52.170721+0000","last_unstale":"2026-02-22T23:21:27.720945+0000","last_undegraded":"2026-02-22T23:21:04.808232+0000","last_fullsized":"2026-02-22T23:21:04.808232+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:53:14.055184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.213379+0000","last_change":"2026-02-22T23:20:54.846662+0000","last_active":"2026-02-22T23:21:04.213379+0000","last_peered":"2026-02-22T23:21:04.213379+0000","last_clean":"2026-02-22T23:21:04.213379+0000","last_became_active":"2026-02-22T23:20:52.171025+0000","last_became_peered":"2026-02-22T23:20:52.171025+0000","last_unstale":"2026-02-22T23:21:29.730323+0000","last_undegraded":"2026-02-22T23:21:04.213379+0000","last_fullsized":"2026-02-22T23:21:04.213379+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:38:59.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025988399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":19,"state":"stale+active+clean","last_fresh":"2026-02-22T23:21:04.189880+0000","last_change":"2026-02-22T23:20:54.173404+0000","last_active":"2026-02-22T23:21:04.189880+0000","last_peered":"2026-02-22T23:21:04.189880+0000","last_clean":"2026-02-22T23:21:04.189880+0000","last_became_active":"2026-02-22T23:20:52.170698+0000","last_became_peered":"2026-02-22T23:20:52.170698+0000","last_unstale":"2026-02-22T23:21:28.726694+0000","last_undegraded":"2026-02-22T23:21:04.189880+0000","last_fullsized":"2026-02-22T23:21:04.189880+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:33:54.336668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":70355,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":143155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738379,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":552960,"data_stored":113056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":120259084291,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149700,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":236,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":241664,"data_stored":69078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":72800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":70355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:22:02.058 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:22:02.213 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:22:02.214 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:22:02.223 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2026-02-22T23:22:02.175993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73058,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":13,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":83348,"kb_used_data":1476,"kb_used_omap":10,"kb_used_meta":81781,"kb_avail":283032172,"statfs":{"total":289910292480,"available":289824944128,"internally_reserved":0,"allocated":1511424,"data_stored":346993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10463,"internal_metadata":83744545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2684,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-9,"num_write_kb":-4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.930607"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2026-02-22T23:21:56.256048+0000","last_change":"2026-02-22T23:21:56.256048+0000","last_active":"2026-02-22T23:21:56.256048+0000","last_peered":"2026-02-22T23:21:56.256048+0000","last_clean":"2026-02-22T23:21:56.256048+0000","last_became_active":"2026-02-22T23:21:56.255728+0000","last_became_peered":"2026-02-22T23:21:56.255728+0000","last_unstale":"2026-02-22T23:21:56.256048+0000","last_undegraded":"2026-02-22T23:21:56.256048+0000","last_fullsized":"2026-02-22T23:21:56.256048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:02:44.796530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-22T23:21:56.256006+0000","last_change":"2026-02-22T23:21:56.256006+0000","last_active":"2026-02-22T23:21:56.256006+0000","last_peered":"2026-02-22T23:21:56.256006+0000","last_clean":"2026-02-22T23:21:56.256006+0000","last_became_active":"2026-02-22T23:21:56.255707+0000","last_became_peered":"2026-02-22T23:21:56.255707+0000","last_unstale":"2026-02-22T23:21:56.256006+0000","last_undegraded":"2026-02-22T23:21:56.256006+0000","last_fullsized":"2026-02-22T23:21:56.256006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:34:40.269352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+undersized","last_fresh":"2026-02-22T23:21:56.253659+0000","last_change":"2026-02-22T23:21:54.248330+0000","last_active":"2026-02-22T23:21:56.253659+0000","last_peered":"2026-02-22T23:21:56.253659+0000","last_clean":"2026-02-22T23:20:51.163086+0000","last_became_active":"2026-02-22T23:21:54.248330+0000","last_became_peered":"2026-02-22T23:21:54.248330+0000","last_unstale":"2026-02-22T23:21:56.253659+0000","last_undegraded":"2026-02-22T23:21:56.253659+0000","last_fullsized":"2026-02-22T23:21:54.246993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:52:49.818130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-22T23:21:56.255618+0000","last_change":"2026-02-22T23:21:56.255618+0000","last_active":"2026-02-22T23:21:56.255618+0000","last_peered":"2026-02-22T23:21:56.255618+0000","last_clean":"2026-02-22T23:21:56.255618+0000","last_became_active":"2026-02-22T23:21:56.255351+0000","last_became_peered":"2026-02-22T23:21:56.255351+0000","last_unstale":"2026-02-22T23:21:56.255618+0000","last_undegraded":"2026-02-22T23:21:56.255618+0000","last_fullsized":"2026-02-22T23:21:56.255618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:10:27.401737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"31'101","reported_seq":121,"reported_epoch":32,"state":"active+undersized+degraded","last_fresh":"2026-02-22T23:21:56.254581+0000","last_change":"2026-02-22T23:21:55.250056+0000","last_active":"2026-02-22T23:21:56.254581+0000","last_peered":"2026-02-22T23:21:56.254581+0000","last_clean":"2026-02-22T23:21:10.577641+0000","last_became_active":"2026-02-22T23:21:55.250056+0000","last_became_peered":"2026-02-22T23:21:55.250056+0000","last_unstale":"2026-02-22T23:21:56.254581+0000","last_undegraded":"2026-02-22T23:21:55.249320+0000","last_fullsized":"2026-02-22T23:21:55.249234+0000","mapping_epoch":30,"log_start":"19'85","ondisk_log_start":"19'85","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":85,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:31.369042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73039,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":100}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":41,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2026-02-22T23:21:56.257567+0000","last_change":"2026-02-22T23:21:56.257567+0000","last_active":"2026-02-22T23:21:56.257567+0000","last_peered":"2026-02-22T23:21:56.257567+0000","last_clean":"2026-02-22T23:21:56.257567+0000","last_became_active":"2026-02-22T23:21:56.255742+0000","last_became_peered":"2026-02-22T23:21:56.255742+0000","last_unstale":"2026-02-22T23:21:56.257567+0000","last_undegraded":"2026-02-22T23:21:56.257567+0000","last_fullsized":"2026-02-22T23:21:56.257567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:26:52.605292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-22T23:21:56.253535+0000","last_change":"2026-02-22T23:21:55.250672+0000","last_active":"2026-02-22T23:21:56.253535+0000","last_peered":"2026-02-22T23:21:56.253535+0000","last_clean":"2026-02-22T23:21:56.253535+0000","last_became_active":"2026-02-22T23:21:55.250294+0000","last_became_peered":"2026-02-22T23:21:55.250294+0000","last_unstale":"2026-02-22T23:21:56.253535+0000","last_undegraded":"2026-02-22T23:21:56.253535+0000","last_fullsized":"2026-02-22T23:21:56.253535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:53:52.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+undersized","last_fresh":"2026-02-22T23:21:56.253603+0000","last_change":"2026-02-22T23:21:54.247887+0000","last_active":"2026-02-22T23:21:56.253603+0000","last_peered":"2026-02-22T23:21:56.253603+0000","last_clean":"2026-02-22T23:20:51.163086+0000","last_became_active":"2026-02-22T23:21:54.247887+0000","last_became_peered":"2026-02-22T23:21:54.247887+0000","last_unstale":"2026-02-22T23:21:56.253603+0000","last_undegraded":"2026-02-22T23:21:56.253603+0000","last_fullsized":"2026-02-22T23:21:54.246814+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:19:54.679416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2026-02-22T23:21:56.255998+0000","last_change":"2026-02-22T23:21:56.255998+0000","last_active":"2026-02-22T23:21:56.255998+0000","last_peered":"2026-02-22T23:21:56.255998+0000","last_clean":"2026-02-22T23:21:56.255998+0000","last_became_active":"2026-02-22T23:21:56.255716+0000","last_became_peered":"2026-02-22T23:21:56.255716+0000","last_unstale":"2026-02-22T23:21:56.255998+0000","last_undegraded":"2026-02-22T23:21:56.255998+0000","last_fullsized":"2026-02-22T23:21:56.255998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:32:42.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":73039,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":143394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":1,"acting":1,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":14,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738379,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":552960,"data_stored":113056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":28,"seq":120259084292,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":668,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":684032,"data_stored":153488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28044973},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149701,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":274432,"data_stored":80449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":73039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":70355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:22:02.224 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2026-02-22T23:22:02.224 INFO:tasks.divergent_priors:delay recovery 2026-02-22T23:22:02.225 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok set_recovery_delay 100000 2026-02-22T23:22:02.296 INFO:teuthology.orchestra.run.trial068.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:22:02.303 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok set_recovery_delay 100000 2026-02-22T23:22:02.373 INFO:teuthology.orchestra.run.trial068.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:22:02.381 INFO:tasks.divergent_priors:revive divergent 2 2026-02-22T23:22:02.381 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noup 2026-02-22T23:22:03.261 INFO:teuthology.orchestra.run.trial068.stderr:noup is set 2026-02-22T23:22:03.273 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T23:22:03.274 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-22T23:22:03.276 INFO:tasks.ceph.osd.2:Started 2026-02-22T23:22:03.276 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:config diff 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:config diff get 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:config get 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:config help [] 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:config set ... 2026-02-22T23:22:03.335 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:22:03.337 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:22:03.337 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-22T23:22:04.530 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:04.525+0000 7f716de7d8c0 -1 Falling back to public interface 2026-02-22T23:22:08.338 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T23:22:08.388 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:08.381+0000 7f716de7d8c0 -1 osd.2 19 log_to_monitors true 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:bluefs debug_inject_read_zeros 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:bluefs files list 2026-02-22T23:22:08.400 INFO:teuthology.orchestra.run.trial068.stderr:bluefs stats 2026-02-22T23:22:08.401 INFO:teuthology.orchestra.run.trial068.stderr:bluestore allocator dump block 2026-02-22T23:22:08.401 INFO:teuthology.orchestra.run.trial068.stderr:bluestore allocator fragmentation block 2026-02-22T23:22:08.401 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:22:08.402 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:22:08.402 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-22T23:22:09.245 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:09.241+0000 7f7161c59640 -1 osd.2 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:22:09.268 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:09.261+0000 7f716a627640 -1 osd.2 35 osdmap NOUP flag is set, waiting for it to clear 2026-02-22T23:22:13.403 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T23:22:13.472 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:22:13.472 INFO:teuthology.orchestra.run.trial068.stdout: "ops": [], 2026-02-22T23:22:13.472 INFO:teuthology.orchestra.run.trial068.stdout: "num_ops": 0 2026-02-22T23:22:13.472 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:22:13.480 INFO:tasks.divergent_priors:delay recovery divergent 2026-02-22T23:22:13.480 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok set_recovery_delay 100000 2026-02-22T23:22:13.541 INFO:teuthology.orchestra.run.trial068.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:22:13.549 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noup 2026-02-22T23:22:13.753 INFO:teuthology.orchestra.run.trial068.stderr:noup is unset 2026-02-22T23:22:13.754 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:13.748+0000 7f7161c59640 -1 osd.2 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:22:13.765 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:epoch 36 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:fsid c2872058-bff0-4756-bb48-18a697ed2e43 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:created 2026-02-22T23:20:36.088536+0000 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:modified 2026-02-22T23:22:13.750379+0000 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:flags nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:crush_version 8 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:full_ratio 0.95 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:backfillfull_ratio 0.9 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:nearfull_ratio 0.85 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:require_min_compat_client luminous 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:min_compat_client jewel 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:require_osd_release squid 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:stretch_mode_enabled false 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.87 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:pool 2 'foo' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 read_balance_score 3.03 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:max_osd 3 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:osd.0 up in weight 1 up_from 26 up_thru 30 down_at 20 last_clean_interval [9,19) v2:10.20.193.68:6801/1667233069 v2:10.20.193.68:6802/1667233069 exists,up 6498852d-91da-4351-9faa-b98ee637096f 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:osd.1 up in weight 1 up_from 28 up_thru 31 down_at 21 last_clean_interval [9,19) v2:10.20.193.68:6805/67143671 v2:10.20.193.68:6806/67143671 exists,up 36291856-799f-464f-9583-431e83696b09 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:osd.2 down in weight 1 up_from 8 up_thru 17 down_at 22 last_clean_interval [0,0) v2:10.20.193.68:6801/4010237467 v2:10.20.193.68:6802/4010237467 exists c5f32d0e-9755-48db-a210-680579d30eb7 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:pg_temp 1.3 [1,0] 2026-02-22T23:22:13.911 INFO:teuthology.orchestra.run.trial068.stdout:pg_temp 1.7 [1,0] 2026-02-22T23:22:13.920 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 26 up_thru 30 down_at 20 last_clean_interval [9,19) v2:10.20.193.68:6801/1667233069 v2:10.20.193.68:6802/1667233069 exists,up 6498852d-91da-4351-9faa-b98ee637096f', 'osd.1 up in weight 1 up_from 28 up_thru 31 down_at 21 last_clean_interval [9,19) v2:10.20.193.68:6805/67143671 v2:10.20.193.68:6806/67143671 exists,up 36291856-799f-464f-9583-431e83696b09', 'osd.2 down in weight 1 up_from 8 up_thru 17 down_at 22 last_clean_interval [0,0) v2:10.20.193.68:6801/4010237467 v2:10.20.193.68:6802/4010237467 exists c5f32d0e-9755-48db-a210-680579d30eb7'] 2026-02-22T23:22:23.922 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:epoch 39 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:fsid c2872058-bff0-4756-bb48-18a697ed2e43 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:created 2026-02-22T23:20:36.088536+0000 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:modified 2026-02-22T23:22:16.758729+0000 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:flags nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:crush_version 8 2026-02-22T23:22:24.078 INFO:teuthology.orchestra.run.trial068.stdout:full_ratio 0.95 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:backfillfull_ratio 0.9 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:nearfull_ratio 0.85 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:require_min_compat_client luminous 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:min_compat_client jewel 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:require_osd_release squid 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:stretch_mode_enabled false 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:pool 2 'foo' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 read_balance_score 2.99 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:max_osd 3 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:osd.0 up in weight 1 up_from 26 up_thru 30 down_at 20 last_clean_interval [9,19) v2:10.20.193.68:6801/1667233069 v2:10.20.193.68:6802/1667233069 exists,up 6498852d-91da-4351-9faa-b98ee637096f 2026-02-22T23:22:24.079 INFO:teuthology.orchestra.run.trial068.stdout:osd.1 up in weight 1 up_from 28 up_thru 38 down_at 21 last_clean_interval [9,19) v2:10.20.193.68:6805/67143671 v2:10.20.193.68:6806/67143671 exists,up 36291856-799f-464f-9583-431e83696b09 2026-02-22T23:22:24.080 INFO:teuthology.orchestra.run.trial068.stdout:osd.2 up in weight 1 up_from 37 up_thru 37 down_at 22 last_clean_interval [8,19) v2:10.20.193.68:6809/1057540422 v2:10.20.193.68:6810/1057540422 exists,up c5f32d0e-9755-48db-a210-680579d30eb7 2026-02-22T23:22:24.088 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 26 up_thru 30 down_at 20 last_clean_interval [9,19) v2:10.20.193.68:6801/1667233069 v2:10.20.193.68:6802/1667233069 exists,up 6498852d-91da-4351-9faa-b98ee637096f', 'osd.1 up in weight 1 up_from 28 up_thru 38 down_at 21 last_clean_interval [9,19) v2:10.20.193.68:6805/67143671 v2:10.20.193.68:6806/67143671 exists,up 36291856-799f-464f-9583-431e83696b09', 'osd.2 up in weight 1 up_from 37 up_thru 37 down_at 22 last_clean_interval [8,19) v2:10.20.193.68:6809/1057540422 v2:10.20.193.68:6810/1057540422 exists,up c5f32d0e-9755-48db-a210-680579d30eb7'] 2026-02-22T23:22:24.089 INFO:tasks.divergent_priors:wait for peering 2026-02-22T23:22:24.089 INFO:tasks.util.rados:rados -p foo put foo /etc/fstab 2026-02-22T23:22:24.089 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo put foo /etc/fstab 2026-02-22T23:22:24.114 INFO:tasks.divergent_priors:killing divergent 2 2026-02-22T23:22:24.115 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T23:22:24.115 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:22:27.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-22T23:22:30.216 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T23:22:30.216 INFO:tasks.divergent_priors:reviving divergent 2 2026-02-22T23:22:30.216 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T23:22:30.217 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-22T23:22:30.219 INFO:tasks.ceph.osd.2:Started 2026-02-22T23:22:30.220 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T23:22:30.280 INFO:teuthology.orchestra.run.trial068.stderr:no valid command found; 10 closest matches: 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:0 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:1 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:2 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:abort 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:assert 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:config diff 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:config diff get 2026-02-22T23:22:30.281 INFO:teuthology.orchestra.run.trial068.stderr:config get 2026-02-22T23:22:30.282 INFO:teuthology.orchestra.run.trial068.stderr:config help [] 2026-02-22T23:22:30.282 INFO:teuthology.orchestra.run.trial068.stderr:config set ... 2026-02-22T23:22:30.282 INFO:teuthology.orchestra.run.trial068.stderr:admin_socket: invalid command 2026-02-22T23:22:30.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T23:22:30.283 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-22T23:22:31.466 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:31.460+0000 7fcc3d56b8c0 -1 Falling back to public interface 2026-02-22T23:22:32.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-22T23:22:35.271 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:35.264+0000 7fcc3d56b8c0 -1 osd.2 39 log_to_monitors true 2026-02-22T23:22:35.287 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T23:22:35.329 INFO:tasks.ceph.osd.2.trial068.stderr:2026-02-22T23:22:35.324+0000 7fcc3534f640 -1 osd.2 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:22:35.350 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:22:35.350 INFO:teuthology.orchestra.run.trial068.stdout: "ops": [], 2026-02-22T23:22:35.350 INFO:teuthology.orchestra.run.trial068.stdout: "num_ops": 0 2026-02-22T23:22:35.350 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:22:55.358 INFO:tasks.divergent_priors:allowing recovery 2026-02-22T23:22:55.358 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq ' 0' 2026-02-22T23:22:55.435 INFO:teuthology.orchestra.run.trial068.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:22:55.443 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 debug kick_recovery_wq ' 0' 2026-02-22T23:22:55.516 INFO:teuthology.orchestra.run.trial068.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:22:55.525 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq ' 0' 2026-02-22T23:22:55.596 INFO:teuthology.orchestra.run.trial068.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:22:55.605 INFO:tasks.divergent_priors:reading divergent objects 2026-02-22T23:22:55.605 INFO:tasks.util.rados:rados -p foo get existing_0 /tmp/existing 2026-02-22T23:22:55.606 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_0 /tmp/existing 2026-02-22T23:22:55.632 INFO:tasks.util.rados:rados -p foo get existing_1 /tmp/existing 2026-02-22T23:22:55.632 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_1 /tmp/existing 2026-02-22T23:22:55.697 INFO:tasks.util.rados:rados -p foo get existing_2 /tmp/existing 2026-02-22T23:22:55.697 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_2 /tmp/existing 2026-02-22T23:22:55.761 INFO:tasks.util.rados:rados -p foo get existing_3 /tmp/existing 2026-02-22T23:22:55.766 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_3 /tmp/existing 2026-02-22T23:22:55.832 INFO:tasks.util.rados:rados -p foo get existing_4 /tmp/existing 2026-02-22T23:22:55.833 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_4 /tmp/existing 2026-02-22T23:22:55.906 INFO:tasks.util.rados:rados -p foo get existing_5 /tmp/existing 2026-02-22T23:22:55.906 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_5 /tmp/existing 2026-02-22T23:22:55.977 INFO:tasks.util.rados:rados -p foo get existing_6 /tmp/existing 2026-02-22T23:22:55.977 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_6 /tmp/existing 2026-02-22T23:22:56.044 INFO:tasks.util.rados:rados -p foo get existing_7 /tmp/existing 2026-02-22T23:22:56.044 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_7 /tmp/existing 2026-02-22T23:22:56.109 INFO:tasks.util.rados:rados -p foo get existing_8 /tmp/existing 2026-02-22T23:22:56.109 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_8 /tmp/existing 2026-02-22T23:22:56.181 INFO:tasks.util.rados:rados -p foo get existing_9 /tmp/existing 2026-02-22T23:22:56.181 DEBUG:teuthology.orchestra.run.trial068:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p foo get existing_9 /tmp/existing 2026-02-22T23:22:56.254 INFO:tasks.divergent_priors:success 2026-02-22T23:22:56.254 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-22T23:22:56.270 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:22:56.270 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:22:56.443 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:22:56.443 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:22:56.453 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2026-02-22T23:22:56.181568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73786,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":84688,"kb_used_data":1872,"kb_used_omap":14,"kb_used_meta":82737,"kb_avail":283030832,"statfs":{"total":289910292480,"available":289823571968,"internally_reserved":0,"allocated":1916928,"data_stored":484883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14947,"internal_metadata":84723101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001109"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332740+0000","last_change":"2026-02-22T23:22:38.332740+0000","last_active":"2026-02-22T23:22:38.332740+0000","last_peered":"2026-02-22T23:22:38.332740+0000","last_clean":"2026-02-22T23:22:38.332740+0000","last_became_active":"2026-02-22T23:22:38.332504+0000","last_became_peered":"2026-02-22T23:22:38.332504+0000","last_unstale":"2026-02-22T23:22:38.332740+0000","last_undegraded":"2026-02-22T23:22:38.332740+0000","last_fullsized":"2026-02-22T23:22:38.332740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:43:17.538497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330669+0000","last_change":"2026-02-22T23:21:56.256006+0000","last_active":"2026-02-22T23:22:38.330669+0000","last_peered":"2026-02-22T23:22:38.330669+0000","last_clean":"2026-02-22T23:22:38.330669+0000","last_became_active":"2026-02-22T23:21:56.255707+0000","last_became_peered":"2026-02-22T23:21:56.255707+0000","last_unstale":"2026-02-22T23:22:38.330669+0000","last_undegraded":"2026-02-22T23:22:38.330669+0000","last_fullsized":"2026-02-22T23:22:38.330669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:34:40.269352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209001+0000","last_change":"2026-02-22T23:22:39.209001+0000","last_active":"2026-02-22T23:22:39.209001+0000","last_peered":"2026-02-22T23:22:39.209001+0000","last_clean":"2026-02-22T23:22:39.209001+0000","last_became_active":"2026-02-22T23:22:39.208660+0000","last_became_peered":"2026-02-22T23:22:39.208660+0000","last_unstale":"2026-02-22T23:22:39.209001+0000","last_undegraded":"2026-02-22T23:22:39.209001+0000","last_fullsized":"2026-02-22T23:22:39.209001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:28:10.268284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330644+0000","last_change":"2026-02-22T23:21:56.255618+0000","last_active":"2026-02-22T23:22:38.330644+0000","last_peered":"2026-02-22T23:22:38.330644+0000","last_clean":"2026-02-22T23:22:38.330644+0000","last_became_active":"2026-02-22T23:21:56.255351+0000","last_became_peered":"2026-02-22T23:21:56.255351+0000","last_unstale":"2026-02-22T23:22:38.330644+0000","last_undegraded":"2026-02-22T23:22:38.330644+0000","last_fullsized":"2026-02-22T23:22:38.330644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:10:27.401737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'102","reported_seq":170,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.345763+0000","last_change":"2026-02-22T23:22:38.345763+0000","last_active":"2026-02-22T23:22:38.345763+0000","last_peered":"2026-02-22T23:22:38.345763+0000","last_clean":"2026-02-22T23:22:38.345763+0000","last_became_active":"2026-02-22T23:22:38.332822+0000","last_became_peered":"2026-02-22T23:22:38.332822+0000","last_unstale":"2026-02-22T23:22:38.345763+0000","last_undegraded":"2026-02-22T23:22:38.345763+0000","last_fullsized":"2026-02-22T23:22:38.345763+0000","mapping_epoch":42,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":95,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:11:06.580319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332768+0000","last_change":"2026-02-22T23:22:38.332768+0000","last_active":"2026-02-22T23:22:38.332768+0000","last_peered":"2026-02-22T23:22:38.332768+0000","last_clean":"2026-02-22T23:22:38.332768+0000","last_became_active":"2026-02-22T23:22:38.332534+0000","last_became_peered":"2026-02-22T23:22:38.332534+0000","last_unstale":"2026-02-22T23:22:38.332768+0000","last_undegraded":"2026-02-22T23:22:38.332768+0000","last_fullsized":"2026-02-22T23:22:38.332768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T11:18:31.302107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.207847+0000","last_change":"2026-02-22T23:21:55.250672+0000","last_active":"2026-02-22T23:22:39.207847+0000","last_peered":"2026-02-22T23:22:39.207847+0000","last_clean":"2026-02-22T23:22:39.207847+0000","last_became_active":"2026-02-22T23:21:55.250294+0000","last_became_peered":"2026-02-22T23:21:55.250294+0000","last_unstale":"2026-02-22T23:22:39.207847+0000","last_undegraded":"2026-02-22T23:22:39.207847+0000","last_fullsized":"2026-02-22T23:22:39.207847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:53:52.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209364+0000","last_change":"2026-02-22T23:22:39.209364+0000","last_active":"2026-02-22T23:22:39.209364+0000","last_peered":"2026-02-22T23:22:39.209364+0000","last_clean":"2026-02-22T23:22:39.209364+0000","last_became_active":"2026-02-22T23:22:39.209062+0000","last_became_peered":"2026-02-22T23:22:39.209062+0000","last_unstale":"2026-02-22T23:22:39.209364+0000","last_undegraded":"2026-02-22T23:22:39.209364+0000","last_fullsized":"2026-02-22T23:22:39.209364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:30:41.079427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330633+0000","last_change":"2026-02-22T23:21:56.255998+0000","last_active":"2026-02-22T23:22:38.330633+0000","last_peered":"2026-02-22T23:22:38.330633+0000","last_clean":"2026-02-22T23:22:38.330633+0000","last_became_active":"2026-02-22T23:21:56.255716+0000","last_became_peered":"2026-02-22T23:21:56.255716+0000","last_unstale":"2026-02-22T23:22:38.330633+0000","last_undegraded":"2026-02-22T23:22:38.330633+0000","last_fullsized":"2026-02-22T23:22:38.330633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:32:42.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":827392,"data_stored":147534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":42,"seq":180388626437,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":756,"kb_used_omap":5,"kb_used_meta":27898,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":774144,"data_stored":186204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6078,"internal_metadata":28567618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":28,"seq":120259084303,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":760,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":778240,"data_stored":186223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28044973},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149712,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":356,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":364544,"data_stored":112456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:22:56.455 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:22:56.605 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:22:56.605 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:22:56.615 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2026-02-22T23:22:56.181568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73786,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":84688,"kb_used_data":1872,"kb_used_omap":14,"kb_used_meta":82737,"kb_avail":283030832,"statfs":{"total":289910292480,"available":289823571968,"internally_reserved":0,"allocated":1916928,"data_stored":484883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14947,"internal_metadata":84723101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001109"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332740+0000","last_change":"2026-02-22T23:22:38.332740+0000","last_active":"2026-02-22T23:22:38.332740+0000","last_peered":"2026-02-22T23:22:38.332740+0000","last_clean":"2026-02-22T23:22:38.332740+0000","last_became_active":"2026-02-22T23:22:38.332504+0000","last_became_peered":"2026-02-22T23:22:38.332504+0000","last_unstale":"2026-02-22T23:22:38.332740+0000","last_undegraded":"2026-02-22T23:22:38.332740+0000","last_fullsized":"2026-02-22T23:22:38.332740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:43:17.538497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330669+0000","last_change":"2026-02-22T23:21:56.256006+0000","last_active":"2026-02-22T23:22:38.330669+0000","last_peered":"2026-02-22T23:22:38.330669+0000","last_clean":"2026-02-22T23:22:38.330669+0000","last_became_active":"2026-02-22T23:21:56.255707+0000","last_became_peered":"2026-02-22T23:21:56.255707+0000","last_unstale":"2026-02-22T23:22:38.330669+0000","last_undegraded":"2026-02-22T23:22:38.330669+0000","last_fullsized":"2026-02-22T23:22:38.330669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:34:40.269352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209001+0000","last_change":"2026-02-22T23:22:39.209001+0000","last_active":"2026-02-22T23:22:39.209001+0000","last_peered":"2026-02-22T23:22:39.209001+0000","last_clean":"2026-02-22T23:22:39.209001+0000","last_became_active":"2026-02-22T23:22:39.208660+0000","last_became_peered":"2026-02-22T23:22:39.208660+0000","last_unstale":"2026-02-22T23:22:39.209001+0000","last_undegraded":"2026-02-22T23:22:39.209001+0000","last_fullsized":"2026-02-22T23:22:39.209001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:28:10.268284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330644+0000","last_change":"2026-02-22T23:21:56.255618+0000","last_active":"2026-02-22T23:22:38.330644+0000","last_peered":"2026-02-22T23:22:38.330644+0000","last_clean":"2026-02-22T23:22:38.330644+0000","last_became_active":"2026-02-22T23:21:56.255351+0000","last_became_peered":"2026-02-22T23:21:56.255351+0000","last_unstale":"2026-02-22T23:22:38.330644+0000","last_undegraded":"2026-02-22T23:22:38.330644+0000","last_fullsized":"2026-02-22T23:22:38.330644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:10:27.401737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'102","reported_seq":170,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.345763+0000","last_change":"2026-02-22T23:22:38.345763+0000","last_active":"2026-02-22T23:22:38.345763+0000","last_peered":"2026-02-22T23:22:38.345763+0000","last_clean":"2026-02-22T23:22:38.345763+0000","last_became_active":"2026-02-22T23:22:38.332822+0000","last_became_peered":"2026-02-22T23:22:38.332822+0000","last_unstale":"2026-02-22T23:22:38.345763+0000","last_undegraded":"2026-02-22T23:22:38.345763+0000","last_fullsized":"2026-02-22T23:22:38.345763+0000","mapping_epoch":42,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":95,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:11:06.580319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332768+0000","last_change":"2026-02-22T23:22:38.332768+0000","last_active":"2026-02-22T23:22:38.332768+0000","last_peered":"2026-02-22T23:22:38.332768+0000","last_clean":"2026-02-22T23:22:38.332768+0000","last_became_active":"2026-02-22T23:22:38.332534+0000","last_became_peered":"2026-02-22T23:22:38.332534+0000","last_unstale":"2026-02-22T23:22:38.332768+0000","last_undegraded":"2026-02-22T23:22:38.332768+0000","last_fullsized":"2026-02-22T23:22:38.332768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T11:18:31.302107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.207847+0000","last_change":"2026-02-22T23:21:55.250672+0000","last_active":"2026-02-22T23:22:39.207847+0000","last_peered":"2026-02-22T23:22:39.207847+0000","last_clean":"2026-02-22T23:22:39.207847+0000","last_became_active":"2026-02-22T23:21:55.250294+0000","last_became_peered":"2026-02-22T23:21:55.250294+0000","last_unstale":"2026-02-22T23:22:39.207847+0000","last_undegraded":"2026-02-22T23:22:39.207847+0000","last_fullsized":"2026-02-22T23:22:39.207847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:53:52.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209364+0000","last_change":"2026-02-22T23:22:39.209364+0000","last_active":"2026-02-22T23:22:39.209364+0000","last_peered":"2026-02-22T23:22:39.209364+0000","last_clean":"2026-02-22T23:22:39.209364+0000","last_became_active":"2026-02-22T23:22:39.209062+0000","last_became_peered":"2026-02-22T23:22:39.209062+0000","last_unstale":"2026-02-22T23:22:39.209364+0000","last_undegraded":"2026-02-22T23:22:39.209364+0000","last_fullsized":"2026-02-22T23:22:39.209364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:30:41.079427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330633+0000","last_change":"2026-02-22T23:21:56.255998+0000","last_active":"2026-02-22T23:22:38.330633+0000","last_peered":"2026-02-22T23:22:38.330633+0000","last_clean":"2026-02-22T23:22:38.330633+0000","last_became_active":"2026-02-22T23:21:56.255716+0000","last_became_peered":"2026-02-22T23:21:56.255716+0000","last_unstale":"2026-02-22T23:22:38.330633+0000","last_undegraded":"2026-02-22T23:22:38.330633+0000","last_fullsized":"2026-02-22T23:22:38.330633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:32:42.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":827392,"data_stored":147534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":42,"seq":180388626437,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":756,"kb_used_omap":5,"kb_used_meta":27898,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":774144,"data_stored":186204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6078,"internal_metadata":28567618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":28,"seq":120259084303,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":760,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":778240,"data_stored":186223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28044973},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149712,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":356,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":364544,"data_stored":112456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:22:56.615 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:22:56.615 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:22:56.768 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:22:56.769 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:22:56.779 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2026-02-22T23:22:56.181568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73786,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":84688,"kb_used_data":1872,"kb_used_omap":14,"kb_used_meta":82737,"kb_avail":283030832,"statfs":{"total":289910292480,"available":289823571968,"internally_reserved":0,"allocated":1916928,"data_stored":484883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14947,"internal_metadata":84723101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001109"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332740+0000","last_change":"2026-02-22T23:22:38.332740+0000","last_active":"2026-02-22T23:22:38.332740+0000","last_peered":"2026-02-22T23:22:38.332740+0000","last_clean":"2026-02-22T23:22:38.332740+0000","last_became_active":"2026-02-22T23:22:38.332504+0000","last_became_peered":"2026-02-22T23:22:38.332504+0000","last_unstale":"2026-02-22T23:22:38.332740+0000","last_undegraded":"2026-02-22T23:22:38.332740+0000","last_fullsized":"2026-02-22T23:22:38.332740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:43:17.538497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330669+0000","last_change":"2026-02-22T23:21:56.256006+0000","last_active":"2026-02-22T23:22:38.330669+0000","last_peered":"2026-02-22T23:22:38.330669+0000","last_clean":"2026-02-22T23:22:38.330669+0000","last_became_active":"2026-02-22T23:21:56.255707+0000","last_became_peered":"2026-02-22T23:21:56.255707+0000","last_unstale":"2026-02-22T23:22:38.330669+0000","last_undegraded":"2026-02-22T23:22:38.330669+0000","last_fullsized":"2026-02-22T23:22:38.330669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:34:40.269352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209001+0000","last_change":"2026-02-22T23:22:39.209001+0000","last_active":"2026-02-22T23:22:39.209001+0000","last_peered":"2026-02-22T23:22:39.209001+0000","last_clean":"2026-02-22T23:22:39.209001+0000","last_became_active":"2026-02-22T23:22:39.208660+0000","last_became_peered":"2026-02-22T23:22:39.208660+0000","last_unstale":"2026-02-22T23:22:39.209001+0000","last_undegraded":"2026-02-22T23:22:39.209001+0000","last_fullsized":"2026-02-22T23:22:39.209001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:28:10.268284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330644+0000","last_change":"2026-02-22T23:21:56.255618+0000","last_active":"2026-02-22T23:22:38.330644+0000","last_peered":"2026-02-22T23:22:38.330644+0000","last_clean":"2026-02-22T23:22:38.330644+0000","last_became_active":"2026-02-22T23:21:56.255351+0000","last_became_peered":"2026-02-22T23:21:56.255351+0000","last_unstale":"2026-02-22T23:22:38.330644+0000","last_undegraded":"2026-02-22T23:22:38.330644+0000","last_fullsized":"2026-02-22T23:22:38.330644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:10:27.401737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'102","reported_seq":170,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.345763+0000","last_change":"2026-02-22T23:22:38.345763+0000","last_active":"2026-02-22T23:22:38.345763+0000","last_peered":"2026-02-22T23:22:38.345763+0000","last_clean":"2026-02-22T23:22:38.345763+0000","last_became_active":"2026-02-22T23:22:38.332822+0000","last_became_peered":"2026-02-22T23:22:38.332822+0000","last_unstale":"2026-02-22T23:22:38.345763+0000","last_undegraded":"2026-02-22T23:22:38.345763+0000","last_fullsized":"2026-02-22T23:22:38.345763+0000","mapping_epoch":42,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":95,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:11:06.580319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332768+0000","last_change":"2026-02-22T23:22:38.332768+0000","last_active":"2026-02-22T23:22:38.332768+0000","last_peered":"2026-02-22T23:22:38.332768+0000","last_clean":"2026-02-22T23:22:38.332768+0000","last_became_active":"2026-02-22T23:22:38.332534+0000","last_became_peered":"2026-02-22T23:22:38.332534+0000","last_unstale":"2026-02-22T23:22:38.332768+0000","last_undegraded":"2026-02-22T23:22:38.332768+0000","last_fullsized":"2026-02-22T23:22:38.332768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T11:18:31.302107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.207847+0000","last_change":"2026-02-22T23:21:55.250672+0000","last_active":"2026-02-22T23:22:39.207847+0000","last_peered":"2026-02-22T23:22:39.207847+0000","last_clean":"2026-02-22T23:22:39.207847+0000","last_became_active":"2026-02-22T23:21:55.250294+0000","last_became_peered":"2026-02-22T23:21:55.250294+0000","last_unstale":"2026-02-22T23:22:39.207847+0000","last_undegraded":"2026-02-22T23:22:39.207847+0000","last_fullsized":"2026-02-22T23:22:39.207847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:53:52.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209364+0000","last_change":"2026-02-22T23:22:39.209364+0000","last_active":"2026-02-22T23:22:39.209364+0000","last_peered":"2026-02-22T23:22:39.209364+0000","last_clean":"2026-02-22T23:22:39.209364+0000","last_became_active":"2026-02-22T23:22:39.209062+0000","last_became_peered":"2026-02-22T23:22:39.209062+0000","last_unstale":"2026-02-22T23:22:39.209364+0000","last_undegraded":"2026-02-22T23:22:39.209364+0000","last_fullsized":"2026-02-22T23:22:39.209364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:30:41.079427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330633+0000","last_change":"2026-02-22T23:21:56.255998+0000","last_active":"2026-02-22T23:22:38.330633+0000","last_peered":"2026-02-22T23:22:38.330633+0000","last_clean":"2026-02-22T23:22:38.330633+0000","last_became_active":"2026-02-22T23:21:56.255716+0000","last_became_peered":"2026-02-22T23:21:56.255716+0000","last_unstale":"2026-02-22T23:22:38.330633+0000","last_undegraded":"2026-02-22T23:22:38.330633+0000","last_fullsized":"2026-02-22T23:22:38.330633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:32:42.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":827392,"data_stored":147534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":42,"seq":180388626437,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":756,"kb_used_omap":5,"kb_used_meta":27898,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":774144,"data_stored":186204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6078,"internal_metadata":28567618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":28,"seq":120259084303,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":760,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":778240,"data_stored":186223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28044973},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149712,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":356,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":364544,"data_stored":112456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:22:56.780 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:56.926 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:22:56.926 INFO:teuthology.orchestra.run.trial068.stdout:{"epoch":43,"fsid":"c2872058-bff0-4756-bb48-18a697ed2e43","created":"2026-02-22T23:20:36.088536+0000","modified":"2026-02-22T23:22:38.328247+0000","last_up_change":"2026-02-22T23:22:37.326067+0000","last_in_change":"2026-02-22T23:21:54.242133+0000","flags":"nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800384,"flags_set":["nodown","noin","noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T23:20:50.267994+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"foo","create_time":"2026-02-22T23:21:01.544700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6498852d-91da-4351-9faa-b98ee637096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":26,"up_thru":41,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6801","nonce":1667233069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6802","nonce":1667233069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6804","nonce":1667233069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6803","nonce":1667233069}]},"public_addr":"10.20.193.68:6801/1667233069","cluster_addr":"10.20.193.68:6802/1667233069","heartbeat_back_addr":"10.20.193.68:6804/1667233069","heartbeat_front_addr":"10.20.193.68:6803/1667233069","state":["exists","up"]},{"osd":1,"uuid":"36291856-799f-464f-9583-431e83696b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":28,"up_thru":42,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6805","nonce":67143671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6806","nonce":67143671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6808","nonce":67143671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6807","nonce":67143671}]},"public_addr":"10.20.193.68:6805/67143671","cluster_addr":"10.20.193.68:6806/67143671","heartbeat_back_addr":"10.20.193.68:6808/67143671","heartbeat_front_addr":"10.20.193.68:6807/67143671","state":["exists","up"]},{"osd":2,"uuid":"c5f32d0e-9755-48db-a210-680579d30eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":37,"last_clean_end":39,"up_from":42,"up_thru":42,"down_at":41,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6809","nonce":1543048644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6810","nonce":1543048644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6812","nonce":1543048644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.68:6811","nonce":1543048644}]},"public_addr":"10.20.193.68:6809/1543048644","cluster_addr":"10.20.193.68:6810/1543048644","heartbeat_back_addr":"10.20.193.68:6812/1543048644","heartbeat_front_addr":"10.20.193.68:6811/1543048644","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2026-02-22T23:21:27.719646+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:20:43.761544+0000","dead_epoch":0},{"osd":1,"down_stamp":"2026-02-22T23:21:28.723603+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:20:43.815984+0000","dead_epoch":0},{"osd":2,"down_stamp":"2026-02-22T23:22:36.324663+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:20:43.136790+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:57.940 INFO:tasks.ceph:Scrubbing osd.0 2026-02-22T23:22:57.940 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:22:58.023 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:22:58.023 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:22:58.023 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:22:58.032 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-22T23:22:58.181 INFO:teuthology.orchestra.run.trial068.stderr:instructed osd(s) 0 to deep-scrub 2026-02-22T23:22:58.191 INFO:tasks.ceph:Scrubbing osd.1 2026-02-22T23:22:58.192 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:22:58.274 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:22:58.274 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:22:58.274 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:22:58.283 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-22T23:22:58.424 INFO:teuthology.orchestra.run.trial068.stderr:instructed osd(s) 1 to deep-scrub 2026-02-22T23:22:58.434 INFO:tasks.ceph:Scrubbing osd.2 2026-02-22T23:22:58.434 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:22:58.506 INFO:teuthology.orchestra.run.trial068.stdout:{ 2026-02-22T23:22:58.507 INFO:teuthology.orchestra.run.trial068.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:22:58.507 INFO:teuthology.orchestra.run.trial068.stdout:} 2026-02-22T23:22:58.519 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-22T23:22:58.664 INFO:teuthology.orchestra.run.trial068.stderr:instructed osd(s) 2 to deep-scrub 2026-02-22T23:22:58.676 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:22:58.819 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:22:58.819 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:22:58.828 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2026-02-22T23:22:58.181795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73786,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":84688,"kb_used_data":1872,"kb_used_omap":14,"kb_used_meta":82737,"kb_avail":283030832,"statfs":{"total":289910292480,"available":289823571968,"internally_reserved":0,"allocated":1916928,"data_stored":484883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14947,"internal_metadata":84723101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001148"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332740+0000","last_change":"2026-02-22T23:22:38.332740+0000","last_active":"2026-02-22T23:22:38.332740+0000","last_peered":"2026-02-22T23:22:38.332740+0000","last_clean":"2026-02-22T23:22:38.332740+0000","last_became_active":"2026-02-22T23:22:38.332504+0000","last_became_peered":"2026-02-22T23:22:38.332504+0000","last_unstale":"2026-02-22T23:22:38.332740+0000","last_undegraded":"2026-02-22T23:22:38.332740+0000","last_fullsized":"2026-02-22T23:22:38.332740+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:43:17.538497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330669+0000","last_change":"2026-02-22T23:21:56.256006+0000","last_active":"2026-02-22T23:22:38.330669+0000","last_peered":"2026-02-22T23:22:38.330669+0000","last_clean":"2026-02-22T23:22:38.330669+0000","last_became_active":"2026-02-22T23:21:56.255707+0000","last_became_peered":"2026-02-22T23:21:56.255707+0000","last_unstale":"2026-02-22T23:22:38.330669+0000","last_undegraded":"2026-02-22T23:22:38.330669+0000","last_fullsized":"2026-02-22T23:22:38.330669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:34:40.269352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209001+0000","last_change":"2026-02-22T23:22:39.209001+0000","last_active":"2026-02-22T23:22:39.209001+0000","last_peered":"2026-02-22T23:22:39.209001+0000","last_clean":"2026-02-22T23:22:39.209001+0000","last_became_active":"2026-02-22T23:22:39.208660+0000","last_became_peered":"2026-02-22T23:22:39.208660+0000","last_unstale":"2026-02-22T23:22:39.209001+0000","last_undegraded":"2026-02-22T23:22:39.209001+0000","last_fullsized":"2026-02-22T23:22:39.209001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:28:10.268284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330644+0000","last_change":"2026-02-22T23:21:56.255618+0000","last_active":"2026-02-22T23:22:38.330644+0000","last_peered":"2026-02-22T23:22:38.330644+0000","last_clean":"2026-02-22T23:22:38.330644+0000","last_became_active":"2026-02-22T23:21:56.255351+0000","last_became_peered":"2026-02-22T23:21:56.255351+0000","last_unstale":"2026-02-22T23:22:38.330644+0000","last_undegraded":"2026-02-22T23:22:38.330644+0000","last_fullsized":"2026-02-22T23:22:38.330644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:10:27.401737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'102","reported_seq":170,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.345763+0000","last_change":"2026-02-22T23:22:38.345763+0000","last_active":"2026-02-22T23:22:38.345763+0000","last_peered":"2026-02-22T23:22:38.345763+0000","last_clean":"2026-02-22T23:22:38.345763+0000","last_became_active":"2026-02-22T23:22:38.332822+0000","last_became_peered":"2026-02-22T23:22:38.332822+0000","last_unstale":"2026-02-22T23:22:38.345763+0000","last_undegraded":"2026-02-22T23:22:38.345763+0000","last_fullsized":"2026-02-22T23:22:38.345763+0000","mapping_epoch":42,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:21:02.182548+0000","last_clean_scrub_stamp":"2026-02-22T23:21:02.182548+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":95,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:11:06.580319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.332768+0000","last_change":"2026-02-22T23:22:38.332768+0000","last_active":"2026-02-22T23:22:38.332768+0000","last_peered":"2026-02-22T23:22:38.332768+0000","last_clean":"2026-02-22T23:22:38.332768+0000","last_became_active":"2026-02-22T23:22:38.332534+0000","last_became_peered":"2026-02-22T23:22:38.332534+0000","last_unstale":"2026-02-22T23:22:38.332768+0000","last_undegraded":"2026-02-22T23:22:38.332768+0000","last_fullsized":"2026-02-22T23:22:38.332768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T11:18:31.302107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.207847+0000","last_change":"2026-02-22T23:21:55.250672+0000","last_active":"2026-02-22T23:22:39.207847+0000","last_peered":"2026-02-22T23:22:39.207847+0000","last_clean":"2026-02-22T23:22:39.207847+0000","last_became_active":"2026-02-22T23:21:55.250294+0000","last_became_peered":"2026-02-22T23:21:55.250294+0000","last_unstale":"2026-02-22T23:22:39.207847+0000","last_undegraded":"2026-02-22T23:22:39.207847+0000","last_fullsized":"2026-02-22T23:22:39.207847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:53:52.482639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:39.209364+0000","last_change":"2026-02-22T23:22:39.209364+0000","last_active":"2026-02-22T23:22:39.209364+0000","last_peered":"2026-02-22T23:22:39.209364+0000","last_clean":"2026-02-22T23:22:39.209364+0000","last_became_active":"2026-02-22T23:22:39.209062+0000","last_became_peered":"2026-02-22T23:22:39.209062+0000","last_unstale":"2026-02-22T23:22:39.209364+0000","last_undegraded":"2026-02-22T23:22:39.209364+0000","last_fullsized":"2026-02-22T23:22:39.209364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:30:41.079427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:38.330633+0000","last_change":"2026-02-22T23:21:56.255998+0000","last_active":"2026-02-22T23:22:38.330633+0000","last_peered":"2026-02-22T23:22:38.330633+0000","last_clean":"2026-02-22T23:22:38.330633+0000","last_became_active":"2026-02-22T23:21:56.255716+0000","last_became_peered":"2026-02-22T23:21:56.255716+0000","last_unstale":"2026-02-22T23:22:38.330633+0000","last_undegraded":"2026-02-22T23:22:38.330633+0000","last_fullsized":"2026-02-22T23:22:38.330633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:20:51.163086+0000","last_clean_scrub_stamp":"2026-02-22T23:20:51.163086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:32:42.557600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":827392,"data_stored":147534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":42,"seq":180388626437,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":756,"kb_used_omap":5,"kb_used_meta":27898,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":774144,"data_stored":186204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6078,"internal_metadata":28567618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":28,"seq":120259084303,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":760,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":778240,"data_stored":186223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28044973},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149712,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":356,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":364544,"data_stored":112456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:22:58.834 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.835 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.835 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.835 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.835 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-22T23:21:02.182548+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=21, tm_sec=2, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.835 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.835 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.836 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.836 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-22T23:20:51.163086+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=20, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=22, tm_sec=56, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:22:58.836 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-22T23:23:18.837 DEBUG:teuthology.orchestra.run.trial068:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:23:18.990 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-22T23:23:18.991 INFO:teuthology.orchestra.run.trial068.stderr:dumped all 2026-02-22T23:23:19.000 INFO:teuthology.orchestra.run.trial068.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-22T23:23:18.183521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73786,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":104,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":84672,"kb_used_data":1872,"kb_used_omap":14,"kb_used_meta":82737,"kb_avail":283030848,"statfs":{"total":289910292480,"available":289823588352,"internally_reserved":0,"allocated":1916928,"data_stored":484883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14947,"internal_metadata":84723101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000994"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:23:01.549867+0000","last_change":"2026-02-22T23:23:01.549889+0000","last_active":"2026-02-22T23:23:01.549867+0000","last_peered":"2026-02-22T23:23:01.549867+0000","last_clean":"2026-02-22T23:23:01.549867+0000","last_became_active":"2026-02-22T23:22:38.332504+0000","last_became_peered":"2026-02-22T23:22:38.332504+0000","last_unstale":"2026-02-22T23:23:01.549867+0000","last_undegraded":"2026-02-22T23:23:01.549867+0000","last_fullsized":"2026-02-22T23:23:01.549867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:01.549798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:01.549798+0000","last_clean_scrub_stamp":"2026-02-22T23:23:01.549798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:38:28.184463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057896800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:23:00.529014+0000","last_change":"2026-02-22T23:23:00.529036+0000","last_active":"2026-02-22T23:23:00.529014+0000","last_peered":"2026-02-22T23:23:00.529014+0000","last_clean":"2026-02-22T23:23:00.529014+0000","last_became_active":"2026-02-22T23:21:56.255707+0000","last_became_peered":"2026-02-22T23:21:56.255707+0000","last_unstale":"2026-02-22T23:23:00.529014+0000","last_undegraded":"2026-02-22T23:23:00.529014+0000","last_fullsized":"2026-02-22T23:23:00.529014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:00.528945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:00.528945+0000","last_clean_scrub_stamp":"2026-02-22T23:23:00.528945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:59.686014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042685500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:23:00.420239+0000","last_change":"2026-02-22T23:23:00.420268+0000","last_active":"2026-02-22T23:23:00.420239+0000","last_peered":"2026-02-22T23:23:00.420239+0000","last_clean":"2026-02-22T23:23:00.420239+0000","last_became_active":"2026-02-22T23:22:39.208660+0000","last_became_peered":"2026-02-22T23:22:39.208660+0000","last_unstale":"2026-02-22T23:23:00.420239+0000","last_undegraded":"2026-02-22T23:23:00.420239+0000","last_fullsized":"2026-02-22T23:23:00.420239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:00.420164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:00.420164+0000","last_clean_scrub_stamp":"2026-02-22T23:23:00.420164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:25:15.292605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:58.462879+0000","last_change":"2026-02-22T23:22:58.462894+0000","last_active":"2026-02-22T23:22:58.462879+0000","last_peered":"2026-02-22T23:22:58.462879+0000","last_clean":"2026-02-22T23:22:58.462879+0000","last_became_active":"2026-02-22T23:21:56.255351+0000","last_became_peered":"2026-02-22T23:21:56.255351+0000","last_unstale":"2026-02-22T23:22:58.462879+0000","last_undegraded":"2026-02-22T23:22:58.462879+0000","last_fullsized":"2026-02-22T23:22:58.462879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:22:58.462822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:22:58.462822+0000","last_clean_scrub_stamp":"2026-02-22T23:22:58.462822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:03:25.957495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028443899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'203","reported_seq":296,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:59.456758+0000","last_change":"2026-02-22T23:22:59.456779+0000","last_active":"2026-02-22T23:22:59.456758+0000","last_peered":"2026-02-22T23:22:59.456758+0000","last_clean":"2026-02-22T23:22:59.456758+0000","last_became_active":"2026-02-22T23:22:38.332822+0000","last_became_peered":"2026-02-22T23:22:38.332822+0000","last_unstale":"2026-02-22T23:22:59.456758+0000","last_undegraded":"2026-02-22T23:22:59.456758+0000","last_fullsized":"2026-02-22T23:22:59.456758+0000","mapping_epoch":42,"log_start":"19'95","ondisk_log_start":"19'95","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'203","last_scrub_stamp":"2026-02-22T23:22:59.456723+0000","last_deep_scrub":"43'203","last_deep_scrub_stamp":"2026-02-22T23:22:59.456723+0000","last_clean_scrub_stamp":"2026-02-22T23:22:59.456723+0000","objects_scrubbed":101,"log_size":108,"log_dups_size":95,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:03:01.345990+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"43'2","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:23:02.551361+0000","last_change":"2026-02-22T23:23:02.551382+0000","last_active":"2026-02-22T23:23:02.551361+0000","last_peered":"2026-02-22T23:23:02.551361+0000","last_clean":"2026-02-22T23:23:02.551361+0000","last_became_active":"2026-02-22T23:22:38.332534+0000","last_became_peered":"2026-02-22T23:22:38.332534+0000","last_unstale":"2026-02-22T23:23:02.551361+0000","last_undegraded":"2026-02-22T23:23:02.551361+0000","last_fullsized":"2026-02-22T23:23:02.551361+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'2","last_scrub_stamp":"2026-02-22T23:23:02.551316+0000","last_deep_scrub":"43'2","last_deep_scrub_stamp":"2026-02-22T23:23:02.551316+0000","last_clean_scrub_stamp":"2026-02-22T23:23:02.551316+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:44:30.628206+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00028637299999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"43'3","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:58.326310+0000","last_change":"2026-02-22T23:22:58.326341+0000","last_active":"2026-02-22T23:22:58.326310+0000","last_peered":"2026-02-22T23:22:58.326310+0000","last_clean":"2026-02-22T23:22:58.326310+0000","last_became_active":"2026-02-22T23:21:55.250294+0000","last_became_peered":"2026-02-22T23:21:55.250294+0000","last_unstale":"2026-02-22T23:22:58.326310+0000","last_undegraded":"2026-02-22T23:22:58.326310+0000","last_fullsized":"2026-02-22T23:22:58.326310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'3","last_scrub_stamp":"2026-02-22T23:22:58.326210+0000","last_deep_scrub":"43'3","last_deep_scrub_stamp":"2026-02-22T23:22:58.326210+0000","last_clean_scrub_stamp":"2026-02-22T23:22:58.326210+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:07:39.977444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000618251,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:23:01.437287+0000","last_change":"2026-02-22T23:23:01.437305+0000","last_active":"2026-02-22T23:23:01.437287+0000","last_peered":"2026-02-22T23:23:01.437287+0000","last_clean":"2026-02-22T23:23:01.437287+0000","last_became_active":"2026-02-22T23:22:39.209062+0000","last_became_peered":"2026-02-22T23:22:39.209062+0000","last_unstale":"2026-02-22T23:23:01.437287+0000","last_undegraded":"2026-02-22T23:23:01.437287+0000","last_fullsized":"2026-02-22T23:23:01.437287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:01.437233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:01.437233+0000","last_clean_scrub_stamp":"2026-02-22T23:23:01.437233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:47:25.819772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T23:22:59.510726+0000","last_change":"2026-02-22T23:22:59.510743+0000","last_active":"2026-02-22T23:22:59.510726+0000","last_peered":"2026-02-22T23:22:59.510726+0000","last_clean":"2026-02-22T23:22:59.510726+0000","last_became_active":"2026-02-22T23:21:56.255716+0000","last_became_peered":"2026-02-22T23:21:56.255716+0000","last_unstale":"2026-02-22T23:22:59.510726+0000","last_undegraded":"2026-02-22T23:22:59.510726+0000","last_fullsized":"2026-02-22T23:22:59.510726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:22:59.510686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:22:59.510686+0000","last_clean_scrub_stamp":"2026-02-22T23:22:59.510686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:54:42.696708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439799,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":73767,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":102,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8247,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":827392,"data_stored":147534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":42,"seq":180388626441,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28676,"kb_used_data":756,"kb_used_omap":5,"kb_used_meta":27898,"kb_avail":94343164,"statfs":{"total":96636764160,"available":96607399936,"internally_reserved":0,"allocated":774144,"data_stored":186204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6078,"internal_metadata":28567618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":28,"seq":120259084307,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":760,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":778240,"data_stored":186223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28044973},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149716,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":356,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":364544,"data_stored":112456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":73767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:23:19.002 DEBUG:teuthology.orchestra.run.trial068:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-22T23:23:19.156 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-22T23:23:19.156 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-22T23:23:19.156 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-22T23:23:19.156 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:23:19.238 INFO:tasks.ceph.osd.0:Stopped 2026-02-22T23:23:19.238 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-22T23:23:19.238 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:23:19.312 INFO:tasks.ceph.osd.1:Stopped 2026-02-22T23:23:19.312 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T23:23:19.312 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:23:19.397 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T23:23:19.397 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-22T23:23:19.397 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-22T23:23:19.398 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:23:19.424 INFO:tasks.ceph.mgr.x:Stopped 2026-02-22T23:23:19.424 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-22T23:23:19.424 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-22T23:23:19.425 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:23:19.477 INFO:tasks.ceph.mon.a:Stopped 2026-02-22T23:23:19.477 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-22T23:23:19.478 DEBUG:teuthology.orchestra.run.trial068:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-22T23:23:19.530 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:23:19.531 DEBUG:teuthology.orchestra.run.trial068:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-22T23:23:19.586 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:23:19.586 DEBUG:teuthology.orchestra.run.trial068:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-22T23:23:19.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:23:19.644 DEBUG:teuthology.orchestra.run.trial068:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-22T23:23:19.695 INFO:tasks.ceph:Archiving mon data... 2026-02-22T23:23:19.697 DEBUG:teuthology.misc:Transferring archived files from trial068:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379/data/mon.a.tgz 2026-02-22T23:23:19.697 DEBUG:teuthology.orchestra.run.trial068:> mktemp 2026-02-22T23:23:19.741 INFO:teuthology.orchestra.run.trial068.stdout:/tmp/tmp.rWSo7nA16q 2026-02-22T23:23:19.741 DEBUG:teuthology.orchestra.run.trial068:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rWSo7nA16q 2026-02-22T23:23:19.811 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod 0666 /tmp/tmp.rWSo7nA16q 2026-02-22T23:23:19.910 DEBUG:teuthology.orchestra.remote:trial068:/tmp/tmp.rWSo7nA16q is 171KB 2026-02-22T23:23:19.918 DEBUG:teuthology.orchestra.run.trial068:> rm -fr /tmp/tmp.rWSo7nA16q 2026-02-22T23:23:19.921 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-22T23:23:19.921 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T23:23:20.017 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-22T23:23:20.017 INFO:tasks.ceph:Archiving crash dumps... 2026-02-22T23:23:20.019 DEBUG:teuthology.misc:Transferring archived files from trial068:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379/remote/trial068/crash 2026-02-22T23:23:20.020 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T23:23:20.069 INFO:tasks.ceph:Compressing logs... 2026-02-22T23:23:20.069 DEBUG:teuthology.orchestra.run.trial068:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:23:20.121 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26801.log 2026-02-22T23:23:20.121 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26217.log 2026-02-22T23:23:20.122 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21445.log 2026-02-22T23:23:20.122 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26801.log.gz 2026-02-22T23:23:20.122 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26217.log.gz 2026-02-22T23:23:20.122 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19016.log 2026-02-22T23:23:20.122 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16082.log 2026-02-22T23:23:20.122 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21445.log: /var/log/ceph/ceph-client.admin.19016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19016.log.gz 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21445.log.gz 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20873.log 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16082.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.16082.log.gz -- 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr: /var/log/ceph/ceph-client.admin.20895.log 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20873.log.gz 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20213.log 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21291.log 2026-02-22T23:23:20.123 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20895.log.gz 2026-02-22T23:23:20.124 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20213.log.gz 2026-02-22T23:23:20.124 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21467.log 2026-02-22T23:23:20.124 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24371.log 2026-02-22T23:23:20.124 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21291.log: /var/log/ceph/ceph-client.admin.21467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21467.log.gz 2026-02-22T23:23:20.124 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21291.log.gz 2026-02-22T23:23:20.124 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20785.log 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22143.log 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24371.log.gz 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20785.log.gz 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18920.log 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21951.log 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22143.log.gz 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.18920.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.18920.log.gz 2026-02-22T23:23:20.125 INFO:teuthology.orchestra.run.trial068.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.21203.log 2026-02-22T23:23:20.126 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21951.log.gz 2026-02-22T23:23:20.126 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19302.log 2026-02-22T23:23:20.126 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21203.log.gz 2026-02-22T23:23:20.126 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19927.log 2026-02-22T23:23:20.126 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20543.log 2026-02-22T23:23:20.126 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19302.log.gz 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22279.log 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19927.log.gz 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21027.log 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20543.log: 0.0%/var/log/ceph/ceph-client.admin.22279.log: -- replaced with /var/log/ceph/ceph-client.admin.20543.log.gz 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22279.log.gz 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19491.log 2026-02-22T23:23:20.127 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19076.log 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21027.log.gz 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21225.log 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19491.log.gz 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19076.log.gz 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20279.log 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20411.log 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21225.log.gz 2026-02-22T23:23:20.128 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20279.log.gz 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20983.log 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26567.log 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20411.log.gz 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20983.log.gz 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19718.log 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26567.log.gz 2026-02-22T23:23:20.129 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26749.log 2026-02-22T23:23:20.130 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19692.log 2026-02-22T23:23:20.130 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19718.log.gz 2026-02-22T23:23:20.130 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20169.log 2026-02-22T23:23:20.130 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26749.log.gz 2026-02-22T23:23:20.130 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19228.log 2026-02-22T23:23:20.130 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19692.log.gz 2026-02-22T23:23:20.131 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16148.log 2026-02-22T23:23:20.131 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20169.log.gz 2026-02-22T23:23:20.131 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22187.log 2026-02-22T23:23:20.131 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19228.log.gz 2026-02-22T23:23:20.131 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20455.log 2026-02-22T23:23:20.131 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16148.log.gz 2026-02-22T23:23:20.132 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21995.log 2026-02-22T23:23:20.132 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22187.log.gz 2026-02-22T23:23:20.132 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22230.log 2026-02-22T23:23:20.132 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20455.log.gz 2026-02-22T23:23:20.132 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21995.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-22T23:23:20.132 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21995.log.gz 2026-02-22T23:23:20.133 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22263.log 2026-02-22T23:23:20.133 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22230.log.gz 2026-02-22T23:23:20.133 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21159.log 2026-02-22T23:23:20.133 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21511.log 2026-02-22T23:23:20.133 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22263.log.gz 2026-02-22T23:23:20.133 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21159.log.gz 2026-02-22T23:23:20.134 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26490.log 2026-02-22T23:23:20.134 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26243.log 2026-02-22T23:23:20.134 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21511.log.gz 2026-02-22T23:23:20.134 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26490.log.gz 2026-02-22T23:23:20.134 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20961.log 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20015.log 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26243.log.gz 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19074.log 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20961.log.gz 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20015.log.gz 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20851.log 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr:gzip/var/log/ceph/ceph-client.admin.19074.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.21819.log 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19074.log.gz 2026-02-22T23:23:20.135 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24167.log 2026-02-22T23:23:20.136 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20851.log: /var/log/ceph/ceph-client.admin.21819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20851.log.gz 2026-02-22T23:23:20.136 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21819.log.gz 2026-02-22T23:23:20.136 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20103.log 2026-02-22T23:23:20.136 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22367.log 2026-02-22T23:23:20.136 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24167.log: /var/log/ceph/ceph-client.admin.20103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24167.log.gz 2026-02-22T23:23:20.136 INFO:teuthology.orchestra.run.trial068.stderr:gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.20103.log.gz --verbose 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr: -- /var/log/ceph/ceph-client.admin.24193.log 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21401.log 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22367.log.gz 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21335.log 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24193.log.gz 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21401.log.gz 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22445.log 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22061.log 2026-02-22T23:23:20.137 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21335.log.gz 2026-02-22T23:23:20.138 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21115.log 2026-02-22T23:23:20.138 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22445.log.gz 2026-02-22T23:23:20.138 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21863.log 2026-02-22T23:23:20.138 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22061.log.gz 2026-02-22T23:23:20.138 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21049.log 2026-02-22T23:23:20.138 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21115.log.gz 2026-02-22T23:23:20.139 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21863.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21863.log.gz 2026-02-22T23:23:20.139 INFO:teuthology.orchestra.run.trial068.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.19253.log 2026-02-22T23:23:20.139 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21049.log.gz 2026-02-22T23:23:20.139 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-22T23:23:20.139 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21555.log 2026-02-22T23:23:20.139 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19253.log.gz 2026-02-22T23:23:20.140 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19380.log 2026-02-22T23:23:20.140 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22419.log 2026-02-22T23:23:20.140 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21555.log.gz 2026-02-22T23:23:20.140 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16202.log 2026-02-22T23:23:20.140 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19380.log.gz 2026-02-22T23:23:20.141 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20741.log 2026-02-22T23:23:20.141 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22419.log.gz 2026-02-22T23:23:20.141 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19458.log 2026-02-22T23:23:20.141 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16202.log.gz 2026-02-22T23:23:20.141 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22112.log 2026-02-22T23:23:20.141 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20741.log.gz 2026-02-22T23:23:20.142 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19458.log.gz 2026-02-22T23:23:20.142 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24345.log 2026-02-22T23:23:20.142 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20719.log 2026-02-22T23:23:20.142 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22112.log.gz 2026-02-22T23:23:20.143 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26723.log 2026-02-22T23:23:20.143 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24345.log.gz 2026-02-22T23:23:20.143 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22281.log 2026-02-22T23:23:20.143 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20719.log.gz 2026-02-22T23:23:20.143 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26723.log.gz 2026-02-22T23:23:20.143 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19490.log 2026-02-22T23:23:20.144 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20323.log 2026-02-22T23:23:20.144 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22281.log.gz 2026-02-22T23:23:20.144 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21599.log 2026-02-22T23:23:20.144 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19490.log.gz 2026-02-22T23:23:20.144 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19770.log 2026-02-22T23:23:20.145 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20323.log.gz 2026-02-22T23:23:20.145 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21005.log/var/log/ceph/ceph-client.admin.21599.log: 2026-02-22T23:23:20.145 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21599.log.gz 2026-02-22T23:23:20.145 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19770.log.gz 2026-02-22T23:23:20.145 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22017.log 2026-02-22T23:23:20.145 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21005.log.gz 2026-02-22T23:23:20.146 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26541.log 2026-02-22T23:23:20.146 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22017.log.gz 2026-02-22T23:23:20.146 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21841.log 2026-02-22T23:23:20.146 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18970.log 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26541.log.gz 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21841.log.gz 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22471.log 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20301.log 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.18970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.18970.log.gz 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22471.log.gz 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24241.log 2026-02-22T23:23:20.147 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20301.log.gz 2026-02-22T23:23:20.148 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21357.log 2026-02-22T23:23:20.148 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16350.log 2026-02-22T23:23:20.148 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24241.log.gz 2026-02-22T23:23:20.148 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21357.log.gz 2026-02-22T23:23:20.148 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21181.log 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16350.log.gz 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19971.log 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21181.log.gz 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26671.log 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19971.log.gz 2026-02-22T23:23:20.149 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25357.log 2026-02-22T23:23:20.150 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21885.log 2026-02-22T23:23:20.150 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26671.log.gz 2026-02-22T23:23:20.150 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.25357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25357.log.gz 2026-02-22T23:23:20.150 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21093.log 2026-02-22T23:23:20.151 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24216.log 2026-02-22T23:23:20.151 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21885.log.gz 2026-02-22T23:23:20.151 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20587.log 2026-02-22T23:23:20.151 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21093.log.gz 2026-02-22T23:23:20.151 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24216.log.gz 2026-02-22T23:23:20.151 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21379.log 2026-02-22T23:23:20.152 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19744.log 2026-02-22T23:23:20.152 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20587.log.gz 2026-02-22T23:23:20.152 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21071.log 2026-02-22T23:23:20.152 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21379.log.gz 2026-02-22T23:23:20.152 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19744.log.gz 2026-02-22T23:23:20.152 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20389.log 2026-02-22T23:23:20.153 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19330.log 2026-02-22T23:23:20.153 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21071.log.gz 2026-02-22T23:23:20.153 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18896.log 2026-02-22T23:23:20.153 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20389.log.gz 2026-02-22T23:23:20.153 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19330.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.19330.log.gz -- 2026-02-22T23:23:20.153 INFO:teuthology.orchestra.run.trial068.stderr: /var/log/ceph/ceph-client.admin.26775.log 2026-02-22T23:23:20.154 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.18896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19567.log 2026-02-22T23:23:20.154 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.18896.log.gz 2026-02-22T23:23:20.154 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20147.log 2026-02-22T23:23:20.154 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26775.log.gz 2026-02-22T23:23:20.155 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26697.log 2026-02-22T23:23:20.155 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19567.log.gz 2026-02-22T23:23:20.155 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20609.log 2026-02-22T23:23:20.155 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20147.log.gz 2026-02-22T23:23:20.155 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19666.log 2026-02-22T23:23:20.155 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26697.log.gz 2026-02-22T23:23:20.156 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20609.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.24267.log -- replaced with /var/log/ceph/ceph-client.admin.20609.log.gz 2026-02-22T23:23:20.156 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-22T23:23:20.156 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20433.log 2026-02-22T23:23:20.156 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19666.log.gz 2026-02-22T23:23:20.156 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20235.log 2026-02-22T23:23:20.157 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24267.log.gz 2026-02-22T23:23:20.157 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20433.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.20433.log.gz --verbose 2026-02-22T23:23:20.157 INFO:teuthology.orchestra.run.trial068.stderr: -- /var/log/ceph/ceph-client.admin.19570.log 2026-02-22T23:23:20.157 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19640.log 2026-02-22T23:23:20.157 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20235.log.gz 2026-02-22T23:23:20.157 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19570.log.gz 2026-02-22T23:23:20.158 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20763.log 2026-02-22T23:23:20.158 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26825.log 2026-02-22T23:23:20.158 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19640.log.gz 2026-02-22T23:23:20.158 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20763.log.gz 2026-02-22T23:23:20.158 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26619.log 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21973.log 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26825.log: /var/log/ceph/ceph-client.admin.26619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26825.log.gz 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26619.log.gz 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18990.log 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20367.log 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21973.log.gz 2026-02-22T23:23:20.159 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.18990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.18990.log.gz 2026-02-22T23:23:20.160 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20521.log 2026-02-22T23:23:20.160 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16250.log 2026-02-22T23:23:20.160 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20367.log.gz 2026-02-22T23:23:20.160 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20521.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20521.log.gz 2026-02-22T23:23:20.160 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22393.log 2026-02-22T23:23:20.161 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-22T23:23:20.161 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16250.log.gz/var/log/ceph/ceph-client.admin.22393.log: 2026-02-22T23:23:20.161 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22393.log.gz 2026-02-22T23:23:20.161 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21753.log 2026-02-22T23:23:20.161 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22269.log 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr: 88.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20191.log 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21753.log.gz 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21621.log 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22269.log.gz 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20191.log.gz 2026-02-22T23:23:20.162 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-22T23:23:20.163 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22165.log 2026-02-22T23:23:20.163 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21621.log.gz 2026-02-22T23:23:20.163 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19883.log 2026-02-22T23:23:20.163 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22165.log.gzgzip 2026-02-22T23:23:20.163 INFO:teuthology.orchestra.run.trial068.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.21731.log 2026-02-22T23:23:20.164 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21775.log 2026-02-22T23:23:20.164 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19883.log.gz 2026-02-22T23:23:20.164 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21247.log 2026-02-22T23:23:20.164 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21731.log.gz 2026-02-22T23:23:20.164 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21775.log.gz 2026-02-22T23:23:20.164 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21423.log 2026-02-22T23:23:20.165 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26314.log 2026-02-22T23:23:20.165 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21247.log.gz 2026-02-22T23:23:20.165 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21423.log.gz 2026-02-22T23:23:20.165 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24405.log 2026-02-22T23:23:20.166 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26314.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T23:23:20.166 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26314.log.gz 2026-02-22T23:23:20.166 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26645.log 2026-02-22T23:23:20.166 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24405.log.gz 2026-02-22T23:23:20.166 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26515.log 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16285.log 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26645.log: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26645.log.gz 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26515.log.gz 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26446.log 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19224.log 2026-02-22T23:23:20.167 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16285.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16285.log.gz 2026-02-22T23:23:20.168 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20829.log 2026-02-22T23:23:20.168 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26446.log.gz 2026-02-22T23:23:20.168 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19224.log.gz 2026-02-22T23:23:20.168 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16226.log 2026-02-22T23:23:20.168 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19905.log 2026-02-22T23:23:20.168 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20829.log.gz 2026-02-22T23:23:20.169 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.16226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26292.log 2026-02-22T23:23:20.169 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16226.log.gz 2026-02-22T23:23:20.169 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20697.log 2026-02-22T23:23:20.169 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19905.log.gz 2026-02-22T23:23:20.169 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19153.log 2026-02-22T23:23:20.170 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26292.log.gz 2026-02-22T23:23:20.170 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20059.log 2026-02-22T23:23:20.170 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20697.log.gz 2026-02-22T23:23:20.170 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18945.log 2026-02-22T23:23:20.170 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19153.log.gz 2026-02-22T23:23:20.171 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19860.log 2026-02-22T23:23:20.171 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20059.log.gz 2026-02-22T23:23:20.171 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.18945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.18945.log.gz 2026-02-22T23:23:20.171 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26380.log 2026-02-22T23:23:20.171 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19993.log 2026-02-22T23:23:20.171 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19860.log.gz 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26380.log.gz 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21533.log 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21577.log 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19993.log.gz 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21533.log.gz 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20653.log 2026-02-22T23:23:20.172 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21577.log.gz 2026-02-22T23:23:20.173 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26593.log 2026-02-22T23:23:20.173 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19042.log 2026-02-22T23:23:20.173 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20653.log: /var/log/ceph/ceph-client.admin.26593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20653.log.gz 2026-02-22T23:23:20.173 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26593.log.gz 2026-02-22T23:23:20.173 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25334.log 2026-02-22T23:23:20.174 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19042.log.gz 2026-02-22T23:23:20.174 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20081.log 2026-02-22T23:23:20.174 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.25334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26358.log 2026-02-22T23:23:20.174 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25334.log.gz 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20807.log 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20081.log.gz 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25308.log 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26358.log.gz 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20807.log.gz 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25281.log 2026-02-22T23:23:20.175 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.25308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20917.log 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25308.log.gz 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.25281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25281.log.gz 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26269.log 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21489.log 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20917.log.gz 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26269.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-22T23:23:20.176 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26269.log.gz 2026-02-22T23:23:20.177 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21489.log.gz 2026-02-22T23:23:20.177 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21929.log 2026-02-22T23:23:20.177 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19949.log 2026-02-22T23:23:20.177 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21929.log.gz 2026-02-22T23:23:20.178 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22086.log 2026-02-22T23:23:20.178 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26402.log 2026-02-22T23:23:20.178 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19949.log: /var/log/ceph/ceph-client.admin.22086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22086.log.gz 2026-02-22T23:23:20.178 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19949.log.gz 2026-02-22T23:23:20.178 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21643.log 2026-02-22T23:23:20.178 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24293.log 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26402.log.gz 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21643.log.gz 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19406.log 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21665.log 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24293.log: /var/log/ceph/ceph-client.admin.19406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24293.log.gz 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19406.log.gz 2026-02-22T23:23:20.179 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26336.log 2026-02-22T23:23:20.180 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21665.log.gz 2026-02-22T23:23:20.180 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26468.log 2026-02-22T23:23:20.180 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20675.log 2026-02-22T23:23:20.180 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26336.log: /var/log/ceph/ceph-client.admin.26468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26336.log.gz 2026-02-22T23:23:20.180 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26468.log.gz 2026-02-22T23:23:20.180 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20499.log 2026-02-22T23:23:20.181 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12506.log 2026-02-22T23:23:20.181 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20675.log: /var/log/ceph/ceph-client.admin.20499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20675.log.gz 2026-02-22T23:23:20.181 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20499.log.gz 2026-02-22T23:23:20.181 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20345.log 2026-02-22T23:23:20.181 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph.tmp-client.admin.12506.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12506.log.gz 2026-02-22T23:23:20.181 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22232.log 2026-02-22T23:23:20.182 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.20345.log: /var/log/ceph/ceph-client.admin.21137.log 2026-02-22T23:23:20.182 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20345.log.gz 2026-02-22T23:23:20.182 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22232.log.gz 2026-02-22T23:23:20.182 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21907.log 2026-02-22T23:23:20.182 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20631.log 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21137.log: /var/log/ceph/ceph-client.admin.21907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21907.log.gz 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21137.log.gz 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21797.log 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20477.log 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20631.log.gz 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21797.log.gz 2026-02-22T23:23:20.183 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21687.log 2026-02-22T23:23:20.184 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20477.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-22T23:23:20.184 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20477.log.gz 2026-02-22T23:23:20.184 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20037.log 2026-02-22T23:23:20.184 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21687.log.gz 2026-02-22T23:23:20.184 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19834.log 2026-02-22T23:23:20.185 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20037.log.gz 2026-02-22T23:23:20.185 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19492.log 2026-02-22T23:23:20.185 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19075.log 2026-02-22T23:23:20.185 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19834.log.gz 2026-02-22T23:23:20.185 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22189.log 2026-02-22T23:23:20.186 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19492.log.gz 2026-02-22T23:23:20.186 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26424.log 2026-02-22T23:23:20.186 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19075.log.gz 2026-02-22T23:23:20.186 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22189.log.gz 2026-02-22T23:23:20.186 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19432.log 2026-02-22T23:23:20.186 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24319.log 2026-02-22T23:23:20.187 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.26424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26424.log.gz 2026-02-22T23:23:20.187 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19432.log.gz 2026-02-22T23:23:20.187 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20125.log 2026-02-22T23:23:20.187 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19155.log 2026-02-22T23:23:20.187 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.24319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24319.log.gz 2026-02-22T23:23:20.187 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22039.log 2026-02-22T23:23:20.188 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20125.log.gz 2026-02-22T23:23:20.188 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19155.log.gz 2026-02-22T23:23:20.188 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19306.log 2026-02-22T23:23:20.188 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.22039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22039.log.gz 2026-02-22T23:23:20.188 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19569.log 2026-02-22T23:23:20.188 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20257.log 2026-02-22T23:23:20.189 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19306.log.gz 2026-02-22T23:23:20.189 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19146.log 2026-02-22T23:23:20.189 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19569.log.gz 2026-02-22T23:23:20.189 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20257.log.gz 2026-02-22T23:23:20.189 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21709.log 2026-02-22T23:23:20.189 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.19146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.19146.log.gz 2026-02-22T23:23:20.190 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21269.log 2026-02-22T23:23:20.190 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20939.log 2026-02-22T23:23:20.190 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21709.log.gz 2026-02-22T23:23:20.190 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21269.log.gz 2026-02-22T23:23:20.190 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20565.log 2026-02-22T23:23:20.191 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20939.log.gz 2026-02-22T23:23:20.191 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21313.log 2026-02-22T23:23:20.191 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.20565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.20565.log.gz 2026-02-22T23:23:20.191 INFO:teuthology.orchestra.run.trial068.stderr:/var/log/ceph/ceph-client.admin.21313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21313.log.gz 2026-02-22T23:23:20.200 INFO:teuthology.orchestra.run.trial068.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-22T23:23:20.250 INFO:teuthology.orchestra.run.trial068.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-22T23:23:20.284 INFO:teuthology.orchestra.run.trial068.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-22T23:23:20.289 INFO:teuthology.orchestra.run.trial068.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-22T23:23:20.290 INFO:teuthology.orchestra.run.trial068.stderr: 2026-02-22T23:23:20.291 INFO:teuthology.orchestra.run.trial068.stderr:real 0m0.176s 2026-02-22T23:23:20.291 INFO:teuthology.orchestra.run.trial068.stderr:user 0m0.482s 2026-02-22T23:23:20.291 INFO:teuthology.orchestra.run.trial068.stderr:sys 0m0.077s 2026-02-22T23:23:20.291 INFO:tasks.ceph:Archiving logs... 2026-02-22T23:23:20.291 DEBUG:teuthology.misc:Transferring archived files from trial068:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379/remote/trial068/log 2026-02-22T23:23:20.292 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T23:23:20.493 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-22T23:23:20.516 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-22T23:23:20.517 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:23:20.603 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-22T23:23:20.614 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-22T23:23:20.614 DEBUG:teuthology.orchestra.run.trial068:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:23:23.612 INFO:teuthology.orchestra.run.trial068.stdout: remote refid st t when poll reach delay offset jitter 2026-02-22T23:23:23.612 INFO:teuthology.orchestra.run.trial068.stdout:============================================================================== 2026-02-22T23:23:23.612 INFO:teuthology.orchestra.run.trial068.stdout: 73.185.182.209 204.117.185.216 2 u 39 64 5 42.653 +1.029 3.896 2026-02-22T23:23:23.612 INFO:teuthology.orchestra.run.trial068.stdout: 152.67.232.7 45.79.82.45 3 u 1 64 17 69.538 +3.102 2.635 2026-02-22T23:23:23.612 INFO:teuthology.orchestra.run.trial068.stdout:*nyc2.us.ntp.li 17.253.2.37 2 u 7 64 17 3.230 +1.985 2.532 2026-02-22T23:23:23.612 INFO:teuthology.orchestra.run.trial068.stdout: ntp.maxhost.io 132.163.97.4 2 u 3 64 3 88.543 -7.790 8.512 2026-02-22T23:23:23.612 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-22T23:23:23.624 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-22T23:23:23.624 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-22T23:23:23.642 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-22T23:23:23.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-22T23:23:23.673 INFO:teuthology.task.internal:Duration was 355.115041 seconds 2026-02-22T23:23:23.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-22T23:23:23.685 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-22T23:23:23.685 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:23:23.711 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-22T23:23:23.711 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial068.front.sepia.ceph.com 2026-02-22T23:23:23.711 DEBUG:teuthology.orchestra.run.trial068:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:23:23.763 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-22T23:23:23.763 DEBUG:teuthology.orchestra.run.trial068:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:23:24.023 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-22T23:23:24.023 DEBUG:teuthology.orchestra.run.trial068:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:23:24.032 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:23:24.033 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:23:24.034 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:23:24.034 INFO:teuthology.orchestra.run.trial068.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:23:24.034 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:23:24.102 INFO:teuthology.orchestra.run.trial068.stderr: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:23:24.104 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-22T23:23:24.120 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-22T23:23:24.121 DEBUG:teuthology.orchestra.run.trial068:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:23:24.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-22T23:23:24.164 DEBUG:teuthology.orchestra.run.trial068:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:23:24.200 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern = core 2026-02-22T23:23:24.209 DEBUG:teuthology.orchestra.run.trial068:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:23:24.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:23:24.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-22T23:23:24.267 INFO:teuthology.task.internal:Transferring archived files... 2026-02-22T23:23:24.267 DEBUG:teuthology.misc:Transferring archived files from trial068:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64379/remote/trial068 2026-02-22T23:23:24.267 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T23:23:24.325 INFO:teuthology.task.internal:Removing archive directory... 2026-02-22T23:23:24.325 DEBUG:teuthology.orchestra.run.trial068:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T23:23:24.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-22T23:23:24.378 INFO:teuthology.task.internal:Not uploading archives. 2026-02-22T23:23:24.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-22T23:23:24.397 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-22T23:23:24.397 DEBUG:teuthology.orchestra.run.trial068:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T23:23:24.409 INFO:teuthology.orchestra.run.trial068.stdout: 1179687 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 23:23 /home/ubuntu/cephtest 2026-02-22T23:23:24.410 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-22T23:23:24.427 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-22T23:23:24.463 INFO:teuthology.run:Summary data: description: rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} duration: 355.1150412559509 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-22T23:23:24.463 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T23:23:24.584 INFO:teuthology.run:pass